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ABSTRACT 


This thesis develops and analyzes a model to identify 
attributes of a successful recruiter. Expert Systems 
software is used to elicit from twenty U.S. Army recruiters 
and instructors, who are experts in the field, 
characteristics associated with recruiter success. An 
interactive computer program based on Quasi-Artificial 
Intelligence (QAI) captured the expert's intuition, 
knowledge, experience, and judgments to create expert 
systems that can be used to select U.S. Army recruiters 
before they attend recruiting school. This study found that 
personal characteristics such as Integrity and Commitment, 
and skills such as Persuading and Listening are 
substantially more important than the types of attributes 


generally used to predict recruiter success. 
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i LNTRODUCTION 


ome OVERVIEW 
1. The Problem 

In the years ahead, many factors could threaten the 
continuation of the All-Volunteer armed forces (AVF). 
Recruiting is expected to become more difficult as a 
declining youth population decreases the pool of potential 
recruits. In addition, in times of prosperity, the military 
will continue to compete with perhaps more attractive 
civilian employment. Finally, budget constraints may make 
it even more difficult to expend the resources necessary to 
attract the desired guantity and guality of recruits. 

The services will increasingly need to maximize the 
efficient utilization of its manpower. An area of prime 
consideration and importance is recruiter selection. 
Recruiting problems are aggravated when the wrong 
individuals are sent to recruiting assignments. The costs 
to the military are considerable in terms of both monetary 
and human resources. For example, one Navy study involving 
1,262 male, active duty canvasser recruiters in 1979 
revealed a drop-out rate of 19 percent, costing almost three 
million dollars in base pay, BAQ, and PCS costs alone. 


[Ref. l:p. 68] 


In addition, the recruiting commands suffer because 
of losses in productivity, and the military as a whole 
suffers because of the loss of petty officer talents in 
positions throughout the operating forces. The military is 
not the sole loser, however. Recruiter selection is based 
on good performance as well as other criteria. The 
individuals selected for recruiting duty are usually some of 
the best the services have to offer with respect to previous 
assignments. If these "successful" senior enlisted 
personnel are not successful on recruiting duty, their 
self-confidence, attitude and motivation will probably 
suffer with their loss of productivity and affect future 
assignments, or cause them to get out of the service earlier 
than anticipated. 

Although improved recruiter selection will not solve 
all the services recruiting problems, it could increase 
productivity and morale because of better recruiter/job 
matches, and reduce turnover and related costs from moving, 
training, and replacing recruiters who are not right for the 
job. This thesis will assist recruiter selection by 
developing a tool that may be useful for reducing the 
selection of personnel that do not become successful 
recruiters. 

2. Literature Review 
An in-depth literature review was conducted ina 


prior masters thesis at the Naval Postgraduate School by 





LCDR Joyce Zellweger on the selection of successful 
recruiters (Ref. 2]. My review will summarize her research, 
emphasize the studies most relevant to identifying 
successful recruiters, and identify strengths and weaknesses 
of each approach. In addition, an annotated bibliography is 
provided in Appendix A to this thesis. 

As the following summary will show, although most 
prior studies presented reasonable hypotheses, sound 
analysis, and interesting conclusions, few of the findings 
were significant. The findings that were significant had 
questionable reliability because they were not cross- 
validated. In others, when cross-validation was attempted, 
original results could not be duplicated. A common problem 
in prior studies was the "criterion problem"--measuring 
recruiter performance in a reliable and valid manner. 
Researchers have used a variety of different performance 
measures, including supervisory ratings, school performance, 
percent of quota achieved, and total number of recruits 
enlisted. Yet, many of these measures are often unreliable 
and of questionable validity. For example, supervisory 
ratings are often based on the individual's reputation 
rather than performance. "Even with the best of intentions, 
supervisors can be influenced by characteristics unrelated 
to job effectiveness." (Ref. 2:pp. 18-19] Other problems 
researchers encountered were lack of information about the 


recruiter's job, and reliable recruiter selection methods 


(since most active duty recruiters are selected 
involuntarily). [Ref.92:p-320] 

The following studies are organized by the source of 
information used to identify successful recruiters: 
interviews, test batteries, assessment centers, and 
personnel file data. 


a. Interviews 





The studies involving interviews provided 
little empirical evidence to test hypotheses pertaining to 
successful recruiter attributes. However, they did provide 
a springboard for additional studies and identified the 
types of personal characteristics and attributes necessary 
for effective recruiter performance. In most cases, 
however, the data obtained were based on opinions, involved 
a biased sample, or were simply a framework for further 
research. 

Some of the attributes of successful 
recruiters identified are: 

- They are "movers," "shakers," and salesmen; 
- They are hungry for success and/or promotion; 


- They are aggressive, want responsibility, and want to 
excel; 


- They can communicate effectively; 


- They are friendly, easygoing, outgoing, sympathetic, 
stable, and sincere; 


- They are ambitious, extroverted, and self-motivated. 


b. Test Batteries 

Most of the studies involving test batteries 
yielded disappointing results primarily because they were 
not cross-validated or simply could not measure recruiter 
performance ina reliable and valid manner. A study 
developed by the Navy Personnel Research and Development 
Center (NPRDC) (Borman, et al., 1976-1986) provides the most 
extensive and promising work in this area. This work has 
evolved through four studies over the past ten years. 

NPRDC began with the development of 
behaviorally-based rating scales which attempted to identify 
improved performance criteria for measuring recruiter 
effectiveness. This first study (Borman, et al., 1976) 
identified more than 800 critical incidents describing 
different facets of effective and ineffective recruiter 
performance (see Table 1). The second (Borman, et al., 
1976) phase involved development and validation of an 
inventory battery to predict Navy and Marine Corps recruiter 
performance. They developed a trial predictor battery that 
included several personality, vocational interest, and 
biographical items and scales. In the third phase (Borman 
et al., 1981), the original test battery was expanded and 
refined. The revised battery was analyzed to determine the 
precision of new items in measuring desired constructs and 
whether they had improved the validity of the original test 


battery. The added items did, in fact, enhance the validity 
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of the old battery's constructs in about half the cases. 
Scales derived from these constructs validly predicted 
recruiter effectiveness. 

NPRDC's final Special Assignment Battery 
measured recruiter potential through a selection composite 
of four sub-scales: selling skills, human relations skills, 
organizing skills, and overall performance. Scores in each 
of these areas were correlated with each recruiter's 
production data. Organizing skills was the only sub-scale 
that was statistically insignificant. 

Several personality constructs correlated 
highly with various aspects of recruiter effectiveness: 

- "Making a good impression" and "Enjoying being the 
center of attention" correlated highest with 
selling skills. 

- "Spontaneity, impulsiveness, ambitious, working hard" 
correlated highest with the human relations skills 


CaAcegory. 


- "Unhappy, lack of confidence" related negatively to 
human relations skills effectiveness. 


- "Order, planning ahead" related well to organizing 
skills. 


- "Leading and influencing others" was the construct 
that correlated most highly in the overall performance 
category. 


- Interests in extroverted, dominant, social, and 
leadership activities and occupations, interests in 
sports and competitive activities, and interests in law 
and political activities correlated highly with 
vocational interests. 

The fourth phase of NPRDC's work, published 


in 1985 (Borman, et al., 1986), strongly confirmed the 
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findings of the earlier studies. In concurrent studies, 
Marine Corps recruiters whose scores were in the top 20 
percent obtained 27 percent more recruits than recruiters in 
the lowest 20 percent. In predictive studies, they obtained 
40 percent more recruits. 
c. Assessment Centers 

The assessment center concept involved using 
trained observers to rate a potential recruiter's 
performance during exercises simulating aspects of the 
recruiter job. Assessors focused on personal 
characteristics such as persuasSiveness, sociability, 
flexibility, and practical judgment. The results were 
successful in predicting recruiter school performance, 
however the concept was based on the assumption that the 
people being rated wanted the job. Since most recruiters 
are assigned involuntarily, the assessment center concept 
proved to be infeasible. 

In 1985 a study was conducted by Weltin, 
et al., attempting to evaluate the usefulness of the ratings 
from the original assessment center for predicting job 
performance as a field recruiter. Each individual had been 
rated by trained assessors in exercises including cold 
calls, interviews, a speech, and an in-basket (work 
prioritization). The assessment center sample included 41 
of 57 soldiers who had taken the original battery in 1981 


and completed the training course. Concurrently, a similar 
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study was conducted on a development center sample of 970 
recruiters who were rated in the center, completed training, 
and had at least one contract their first year on the job. 
However, assessors were not trained and instructor ratings 
on telephone and interviewing techniques were not available. 

The results of the assessment center ratings 
showed low correlations with job performance. Furthermore, 
in the development center sample, only the cold call 
interview and speech exercises were significant. A stepwise 
regression was performed on the development center sample, 
using job performance as the dependent variable, with 
ratings, training grades, and other predictors used as the 
independent variables. The results indicated that 
productivity of the recruiter's battalion was the single 
most important factor in predicting job performance. 

dad. Personnel File Variables 

The studies in this area also revealed 
attributes that successful recruiters possess. However, 
most were not cross-validated or failed in cross-validation 
and therefore have limited usefulness. 

The best study was conducted in 1983 by Elig, 
et al., which used the Enlisted Master File (EMF) and the 
Military Enlistment Processing Station Reporting System 
(MRS) as their data source to acquire information on recruit 
characteristics. The sample consisted of 552 male and 60 


female Army recruiters on recruiting duty during fiscal year 
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1979. Descriptive statistics were used to analyze 
Characteristics of recruiters. Those characteristics which 
correlated with contract production were identified using 
analysis of covariance techniques. Results indicated that 
opportunity bias (District Recruiting Command's Average 
Production) explained 32 percent of the variance in 
productivity. The remaining variance was believed to have 
been a result of unmeasured opportunity bias. Some of their 
findings, significant to at least the .01 level were as 
follows: 


- Recruiters with post-secondary education recruited 
better educated, but lower AFQT, males. 


- Recruiter AFQT's correlated positively with recruit 
AFQT's in its "prime" market (high school diploma 
graduate and senior males), but had little impact on 
females or non-high school graduates. 


- Recruiter gender had no effect. 


- Older recruiters contracted more male and fewer female 
recruits than younger recruiters. 


- In total production, younger males out-performed older 
males and their female counterparts, while older females 
out-performed all others. 

- Higher ranking recruiters achieved success in the high 
school diploma graduates and senior males market by 
contracting more low AFQT (category IV) recruits than 
lower ranking recruiters. 

- Black recruiters enlisted the most Blacks, Hispanic 
recruiters enlisted the most Hispanics, whites the most 
whites, etc. 

e. Summary 
Most of the past research on recruiter 


selection suffered from poor criterion measurement, lack of 
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knowledge of the recruiter job, or failure of results to 
remain significant upon cross-validation. As a result, 
findings of many of these studies are of questionable value. 

Encouraging results in recruiter selection 
research was found in the study conducted by NPRDC on the 
Special Assignment Battery and the assessment center 
concept. However, both methods are very costly, which 
reduces their potential for application. 

The Special Assessment Battery proved to be 
highly valid when cross-validated on a sample of Marine 
Corps recruiters. The characteristics which appeared to be 
associated with recruiter success were background and 
personality characteristics, and interest patterns. 

However, the battery is very lengthy and costly to 
administer. Furthermore, if future non-volunteer recruiters 
who took the battery believed their scores would result ina 
recruiting assignment, potential sabotage could reduce the 
tool's usefulness. To get around this, personnel could be 
required to complete the battery well before a time when 
they would associate it with recruiting duty, but these 
would impose even greater costs on the military. [{Ref. 2: 
p. 64] 

The assessment center concept is used in the 
military as a part of recruiter training to indoctrinate and 
familiarize recruiters with what to expect on their way to 


the field. But assessment centers are also costly. Those 
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who do not complete the training are transferred to other 
jobs, while the job in the field is left empty until another 
person can be identified and complete the training. In 
addition to wasted transfer dollars caused by the 
unnecessary moves, other types of hardships may arise for 
the member or his/her family. [Ref. 2:pp. 64-65] 

Several studies attempted to identify passive 
recruiter selection procedures to identify people who would 
most likely become successful recruiters before being 
assigned to recruiting duty. Some of the personal and 
background characteristics were statistically significant, 
but the relative importance of these characteristics in 
recruiter selection could not be determined. Budget cuts 
and increasing numbers of non-volunteer recruiters make 
passive selection procedures more important, but significant 
research questions remain unanswered. freti.82. p.m 5 | 

f. Purpose of Thesis 

Characteristics believed to be related to 
successful recruiting have been identified and summarized in 
Table 2, but which of these characteristics is most 
important? How many prospective recruiters possess all of 
the characteristics believed to be part of the successful 
recruiter profile? If one person has some of these 
characteristics, and another person has some of the others, 


who do we choose? [Ref. 2:pp. 65-66] 
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TABLE 2 
SUMMARY OF CHARACTERISTICS RELATED TO RECRUITER SUCCESS 


Plans Ahead 
Uses Systematic Approach in Prospecting 
Knowledgeable About Recruiting 
Sales Experience 





Age (older if female, younger if male) 
Marital Status 
Paygrade 
Length of Service 
AFQT Scores 


Verbal Fluency 
Persuasiveness 
Communicates Effectively 


Self-Motivated 
Ambitious 
Desire to Excel 
Aggressive 
Dominant 
Confident 
Enthusiastic 
Positive 
Mature 
Financially Stable 
Extroverted 
Enjoys Working with Others 
Spontaneous 
Influences Others 


Well Groomed 


The failure of empirical analysis to 
successfully predict the kind of person who will become a 
successful recruiter may stem from an inability to invoke 
heuristics into "conventional" computer systems. Heuristics 
involve hunches, educated guesses, and rules of thumb which 
are based on experience and knowledge of underlying 


principles. Perhaps the missing link to understanding the 
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profile of a successful recruiter is the judgement of an 
expert who has worked in this area and knows better than 
anyone, innately, what it takes to be a successful 
recruiter. 

Therefore, this thesis will attempt to develop 
a tool to assist in recruiter selection that may be useful 
in identifying individuals with high likelihood of being 
successful recruiters. 

The next chapter describes a methodology for 
making decisions about the relative importance of the 
characteristics believed to be important in the profile of a 
successful recruiter. Chapter III will analyze the 
similarities and differences of the expert systems created 
by each of the experts. Chapter IV will then summarize my 
findings and provide recommendations for future work in 


meeritter selection. 


Ine, 


Il. METHODOLOGY 


The goal of this thesis is to identify the most 





important factors and their relative importance for 
identifying individuals with high likelihood of being 
successful recruiters. To accomplish this, I am going to 
improve the present recruiter model and apply expert systems 
analysis to derive a recruiter selection model for reserve 
Army recruiters. Previous attempts to assess characteris-~- 
tics that can be used to reliably select successful 
recruiters has suffered from the deficiencies indicated in 
the last chapter. The expert systems approach will attempt 
to "fill in the gaps" of previous studies and provide the 
military with a tool to improve recruiter selection 


procedures. 


A. ARTIFICIAL INTELLIGENCE AND EXPERT SYSTEMS 

1. Background and Definition 

The science of Artificial Intelligence (AI) has 

grown significantly in the past few years, particularly in 
commercial applications. It began in the 1950s evolving 
from the fields of mathematics and philosophy and the desire 
of cognitive psychologists to understand human memory and 
reasoning. The first computers were able to formalize and 
emulate many procedural activities. Since intelligence 


involves manipulation of symbols, much of AI is based not on 
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calculating optimal answers, but on the use of heuristics 
that can provide quick, satisfactory answers. [Ref. 3] 

There are many definitions of AI. A scientist or 
research specialist may define it as the study of ideas 
which enable computers to do things that make people seem 
intelligent. A more goal-oriented definition is making 
machines "smart" so they will become more useful and 
understand the principles which make intelligence possible. 
An applications definition is a field of computer problem 
solving. A useful definition for current purposes is an 
effort to develop computer systems that can approximate a 
human's ability to reason and decide. [Ref. 4] 

All areas of AI are involved in the extraction or 
generation of information and in understanding the 
surrounding environment (complex set of data or goals). One 
of the most significant efforts in AI has been in the area 
of expert or Knowledge-based systems. Expert systems model 
complex situations in one specific domain and provide 
conclusions based on a set of rules that have been reduced 
from the knowledge and experience of people who have 
expertise in a given functional area (domain experts) 

[Ref. 5]. Knowledge engineers attempt to elicit and 
formalize problem domain information from human experts in 
the field through interviews and test cases. The knowledge 


engineer becomes acquainted with the facts, identifies the 
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concepts, and develops and codifies the heuristics that the 
expert steps through while solving a problem [Ref. 4]. 

Expert systems attempt to provide an answer to the 
age-old problem of knowledge transfer. That is, while still 
lacking many of the characteristics of true human expertise, 
expert system technology may make the skills of an expert 
available to a broad population who are not experts. (An 
expert is someone who has developed more knowledge in a 
particular subject area than most people in the same field, 
and who can use that knowledge to work more efficiently and 
effectively.) In addition, once the expert system is 
developed, technicians can maintain it without being experts 
themselves. This is particularly useful because the 
know-how developed over years of experience and concentrated 
effort will not be lost when the expert dies [Ref. 6]. 

As shown in Figure 1, the basic parts of an expert 
system are: 

a. Knowledge Base. The most common method of represent- 
ing knowledge is through the rules about a specific 
domain in the form of facts and heuristic rules. 

Facts are known rules about the domain of information, 
while heuristics are those rules of good/better 
judgment developed through experience and trial-and- 
error methods by domain experts while solving problems 
(Ref. 5]. The knowledge base also includes some of 
the system's programs. These programs manipulate the 
symbols which represent the facts and rules of the 
knowledge domain. The computer follows a few simple 
procedures, such as searching, matching, separating, 
joining, substituting, and deleting when processing 
the data or symbols in order to find a solution to the 
problem. This is where AI programming is so differ- 


ent from the conventional program approach of follow- 
ing specific step-by-step procedures. In AI, the 
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Figure 1. Components of an Expert System 


distinction between the facts (data) and the program 
is blurred [Ref. 4]. 


Inference Engine. The program which controls the 
evaluation of a problem and evaluates the rules in the 
knowledge base to define a solution [Ref. 5]. 


Language Processor. Allows a user to interface with 
the system to enter or retrieve data [Ref. 4]. It 
parses and interprets questions from the user and then 
formats the question for the inference engine. 
Finally, when an answer is reached, the language 
processor formats the information for the user [Ref. 
5). 


Knowledge Acquisitioner. The person who ensures that 
the knowledge base has correct and current information 
and that the rules work together properly [Ref. 5]. 
Knowledge acquisition is very difficult because human 
knowledge is complex and can be messy and ill- 
formulated; humans find it extremely difficult to 
articulate the knowledge they have and how they use 
that Knowledge to solve problems; the more expert a 
person becomes, the more 'unconscious' his/her 
knowledge becomes; the data from knowledge acquisition 
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techniques need careful, even skilled, interpretation 
as to what underlying Knowledge may be inferred from 
them; and the techniques which have been developed are 
still only poorly understood, not very robust, and 
often have very limited applicability [Ref. 7]. 

Once the knowledge engineer has collected and 
assimilated the data, he/she must choose from among several 
methods how best to represent the expert's knowledge in the 
system. Knowledge representation, or representing facts and 
rules about a knowledge domain, can be accomplished through 
one or a combination of approaches. Three of the most 
common techniques are Semantic Networks, Frames, and 
Production Rules. 

Semantic Networks were originally proposed in 1966 
as a model of the human memory. They were devised to 
represent relationships among various data elements that are 
found in memory. A common representation scheme consists of 
a tree-like structure of nodes which represent concepts and 
links or branches between the nodes which, by being linked, 
represent attributes that can be "inherited" down the link. 
The best characterization of these links is the "is-a" 
statement. For example, a blue-jay is-a bird, a bird is-a 
mammal. This type of structure leaves room for expansion of 
inheritance characteristics or attributes at each node. The 
links containing the attributes are called by a variety of 
names such as has-a, part-of, or has-part. Thus, attributes 


can be inferred from nodes to which they are linked. An 


example illustrating this idea is a bird has-a wing, a 
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blue-jay is-a bird, therefore, a blue-jay has-a wing. The 
advantage of this technique lies in its excellent ability to 
Poreray relationships: 

The Frames technique is basically a data structure 
similar to a template with "slots" reserved for different 
attributes, and the frame description has all the basic 
information including relationships. A "cat" frame could 
include a description of status (pet), name ("Tinker"), 
breed (siamese), and color (brown and white). Another 
person would also have a frame with the same generic 
information. Both frames would be linked to the conceptual 
"cat" frame where various attributes such as name, breed, 
and physical attributes are contained in "slots". Each slot 
specifies a value or range of values for each attribute. 
This technique represents objects by standard attributes and 
relationships to other frames. The advantage of this 
technique is in the amount of knowledge that can be stored 
about the attributes and relationships of the object in 
question. 

The Production Rules technique is the most popular 
in building expert systems. It was developed in 1972 as a 
model for human cognition. The knowledge base consists of 
an IF statement which spells out a pattern or condition, and 
an action part designated by a THEN statement which 
elaborates an action to be taken or a consequence once the 


condition has been satisfied. For example, IF the cruise 
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control is disengaged and/or the accelerator is released, 
THEN the car will slow down. The condition can be refined 
by adding more qualifications, which makes it particularly 
appealing for use in a military expert system since the 
rules can be very explicit. The advantage of this technique 
is its ability to represent procedural knowledge. 

The most difficult part of this whole process is to 
capture the human thought process. The human mind can 
understand statements which are merely inferred and 
appropriately weigh or associate incoming information with 
past experience. But, how can this be programmed into a 
computer? One of the methods used to weigh confidence 
levels in expert systems is the assignment of confidence 
factors which are usually attached to the rules. The 
reasoning mechanism used in the inference engine is chosen 
by the knowledge engineer and is usually of two types: 
forward chaining or backward chaining. 

Forward chaining attempts to achieve a goal given an 
initial state. It is said to be data driven. Backward 
chaining works backward from a hypothesis to seek the 
evidence that will support it. It is said to be goal driven 
(Ref. 4]. Forward chaining seeks to identify all rules 
whose IF portions are true, then uses the THEN portions of 
those rules to find other rules which are also true. [In 
other words, when trying to solve a problem, this method 


takes a kind of trial and error approach through the 
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information--both permanent facts and rules that have been 
built in, and facts supplied by the person using the system 
pertaining to the problem at hand. 

On the other hand, backward chaining seeks to 
satisfy a stated goal by seeking rules in which the THEN 
portion matches the goal, then seeks other rules whose THEN 
portion matches the IF portion of the rule that it 
satisfies. It starts from the desired result and works 
backward through the rules in its possession, and considers 
only those that are relevant to the goal [Ref. 5]. 


2. Evolution of an Expert System 


a. Identification (Determining Problem 
Characteristics) 


The expert and the knowledge engineer work 
together to identify the problem and define its scope. They 
also identify the participants in the development process 
(additional experts), determine the resources needed and 
decide on the goals and objectives of building the expert 
system. 


b. Conceptualization (Finding Concepts or Represent 
Knowledge) 


The expert and the knowledge engineer explicate 
the key concepts, relationships, and information-flow 
characteristics needed to describe the problem-solving 
process in the given domain. They also specify sub-tasks, 
strategies, and constraints related to the problem-solving 
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c. Formalization (Designing Structures to 
Organizational Knowledge) 


The knowledge engineer must select the language, 
and with the help of the expert, represent the basic 
concepts and relationships within the language framework. 

In other words, map the key concepts into a formal 
representation. 


ad. Implementation (Formulating Rules that Embody 
Knowledge) 


The knowledge engineer combines and reorganizes 
the formalized knowledge to make it compatible with the 
information flow characteristics of the problem. The 
resulting set of rules and control structures associated 
with them define a prototype program which is capable of 
being executed and tested. 


e. Testing (Validating Rules that Embody 
Knowledge) 


The performance of the prototype program must 
then be evaluated and revised to conform to the standards of 
excellence defined by the experts in the problem domain. 

The most important goal in expert systems' work 
is to attain the high level of performance that a human 
expert would achieve in the same task. The quality of 
reasoning in an expert system is based on the accessibility 
of relevant facts and principles as well as on completeness 
of the inference procedure and efficacy of its 
implementation. One way to determine if the expert system 


is successful is to examine its ability to reason about its 
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own processes and be able to satisfy another expert as to 
the soundness of the reasoning sequence. Explanation of 
this process is usually associated with some form of tracing 
of rules that are used during the course of a problem- 
solving session, and reconstructing a rational line of 
argument built out of the fundamental principles of the 
doenain [Ref. 7]. 
3. Types of Expert Systems 

Most expert system applications fall into a few 
distinct types. The following generic categories will 
briefly describe the problem each addresses: 


a. Interpretation--Infers situation descriptions from 
sensor data. 


b. Prediction--Infers likely consequences of given 
Situations. 


c. Diagnosis--Infers system malfunctions from 
observables. 


dad. Design--Configuring objects under constraints. 
e. Planning--Designing actions. 


f. Monitoring--Comparing observations to plan 
vulnerabilities. 


g. Debugging-~-Prescribing remedies for malfunctions. 


h. Repair--Executing a plan to administer a prescribed 
remedy. 


i. Instruction--Diagnosing, debugging, and repairing 
student behavior. 


j. Control--Interpreting, predicting, repairing, and 
monitoring system behaviors. [Ref. 8] 
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The following section will present some examples of 
commercial and military applications of expert systems 
utilizing some of these types. 


4. Applications of Expert Systems 


a. Commercial 

(1) Leukemia Diagnosis. An expert system was 
developed to interpret the results of laboratory tests in 
diagnosing leukemia. "Knowledge" from an expert was 
extracted from transcripts of sessions he gave during a 
running commentary while he made diagnoses on 67 samples 
from various hospitals. The EMYCIN system (an expert 
system) was translated into Edinburgh University DEC-10 
PROLOG to allow the rules making each conclusion to be 
grouped together and analyzed. There were 100 test cases of 
which the system gave satisfactory answers in 70 (70%). 
Simple or hidden errors in rules were corrected and the 
performance improved to 85/90 (94%). This was a small 
system, however, handling only orthodox cases. A high 
quality system must handle rare cases and would require a 
more comprehensive representation of the domain. 

Conclusion: Expert systems offer the hope 

of solving problems as well as human experts. However, the 
modest success in this system does not mean the same success 
will result ina large system. The important factor 
appeared to be in quality control at all stages in the 


development of the expert system [Ref. 9]. 


30 





(2) Automatic Speech Recognition. An expert 


system incorporating phonetic/phonological and lexical 
knowledge was developed by the Speech Group in the Human- 
Computer Interface Research Unit at Leicester Polytechnic. 
The spectrogram was chosen as the initial representation of 
the acoustic wave-form of speech in which communication of 
this Knowledge could be achieved. The expert system used 
PROLOG rules to interpret the spectrogram in terms of both 
phonemes and words. A spectrogram provides a visual 
representation of acoustic waves in terms of the intensity 
of energy presented at each frequency over a period of time 
The program was encoded by an expert using six hundred 
spectrograms produced by a male and a female speaker. It 
was tested on twenty spectrograms of continuous speech from 
four different speakers, two males and two females. The 
results were analyzed in terms of the percentages of 
phonemes correctly identified, those confused with other 
phonemes, and those phonemes which were simply missed. The 
overall phoneme recognition rate was an impressive 62%. 
Conclusion: This approach can achieve more 
than 60% correctly recognized phonemes from unknown 
utterances on a speaker-independent basis and is therefore 
definitely worth pursuing as part of the research into 
speaker-independent unrestricted automatic speech 


recognition [Ref. 10]. 
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(3) Taxation. The development of ACCI, a 
prototype expert system in the area of taxation known as 
"apportionment of close companies' income," incorporates tax 
legislation explicitly, together with the control structures 
relating to the inter-dependency of statutes. This allows 
the tax inspector to obtain advice in an accurate, cost- 
effective and natural way. One of the primary problems in 
this area has been a lack of experts to go around to perform 
this necessary auditing function. The main benefits of this 
system are that it alleviates the problem of a lack of human 
experts in this area of taxation, and information can be 
obtained much more efficiently and accurately. ADVISOR was 
the expert system used, which meant that the inference 
engine of the system was already provided. The knowledge 
for this system was obtained from four sources: Statutes; 
training notes; the tax inspector's "field manual"; and 
interviews with an expert. The knowledge base was built 
through a process of stepwise refinement. That is, a piece 
of legislation is first considered and then translated into 
English-like rules which are subsequently translated into 
ADVISOR rules. Then, the rules are ordered in a structure 
relevant to the application at hand. The results of this 
system were very impressive. Consultations that were taking 
the tax inspector up to half a day to complete were reduced 


to only a few minutes with ACCI. 
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Conclusions: ACCI relieves the tax 
inspector of constantly referring to a manual for assistance 
and provides accurate advice quickly and easily. ACCI 
therefore demonstrates the effective use of expert systems 
technology in solving problems where there is a shortage of 
human experts [Ref. 11}. 

beet atany 

(1) Budget Analysis. Department of Energy 

(DOE) researchers at Oak Ridge National Laboratory in Oak 
Ridge, Tennessee developed one of the first expert systems 
applications for the government's budget process. It was 
created for the Naval Sea Systems Command (NAVSEA) as a 
prototype system which mimics the decision-making 
capabilities of budget analysts. The expert system is 
called BANS, for Budget Analysts, and operates on IBM 
personal computers. The NAVSEA budgeting branch analysts 
are responsible for management and control of all aspects of 
the budget for a group of 28 activities. Each year, they 
have to review several thousand pages of budget requests, 
identify areas with increases and provide justifications for 
those increases within a two-week period. In an initial 
test of the BANS system, a budget analysis that normally 
takes two to three hours for an analyst to perform took BANS 
only fifteen minutes. 

Conclusion: The system proved to work . 


faster than the analysts, but the goal is for greater 
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consistency in decision making. BANS is still in the : 
testing stage and will be used to analyze the same problems 
analysts handle. The results will be compared and the 
system is expected to be fully operational as a decision 
support tool in »the fallwof 1987 [Ref. 12); 

(2) Steam Propulsion System. One of the first 
AI efforts in the area of training was the Steamer Project 
conducted by the Naval Personnel Research and Development 
Center (NPRDC) in San Diego, California. The Steamer 
Project was a research effort concerned with implementing 
intelligent computer based training. A steam propulsion 
system 1s a very complex physical system, and the propulsion 
spaces take up about one-third of the space in most Navy 
ships. Operating the plant requires a supervisor and about 
sixteen to twenty-five people under extremely arduous 
circumstances. It takes years of instruction and experience 
to become competent in this area. The Steamer Project has 
been used as a training aid at Great Lakes Training Center 
and Coronado Island, San Diego, California with very 
encouraging results. Preliminary results show that 
personnel respond positively to the program and can learn in 
a much shorter period of time. 

Conclusion: The possibility of reducing 

the training cycle and maintenance of proficiency through 
realistic training models is extremely attractive to the 


military services. [Ref. 13] 
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Advantages and Disadvantages of Expert Systems 


a. Advantages 


Flexibility of Expression. Expert systems can embody 
rules-of-thumb that practitioners tend to carry 
around in their heads but never write down. 


Human-like Processing. Expert systems operate terms 
and concepts that an individual user can feel 
affinity with and at the level of rules and facts, 
and the relationships between them, rather than at 
the level of program steps. 


Ease of Expression. Expert systems are much more 
easily understood by practitioners in the domain who 
may not be very computer-literate. 


Uncertainty. Uncertainty and contradictory 

evidence are handled in a natural way in 
plausible-inference systems. So, those areas where 
there is incomplete knowledge and where judgement is 
needed, which is so prevalent in real life, can be 
dealt with in expert systems. 


Checklist. Expert systems do not forget, unlike 
people, and can reliably pose all relevant questions 
and act like a checklist. 


Refining Expertise. Most experts admit to having 
gaps in their knowledge and expert systems can help 
identify where these lie. 


Communication Medium. Expert systems can aid in 
searching and processing a huge collection of rules 
ana other information. In addition, it can form the 
medium for sharing experiences between two 
institutions. [Ref. 7] 


Reduce Costs. Extracting knowledge from experts and 
translating it in computer form can greatly reduce 
the costs of knowledge reproduction and exploitation. 


Performs difficult tasks quickly and at expert levels 
of performance. 


Employs self-knowledge to reason about its own 
inference processes and can therefore provide 
explanations or justifications for conclusions it 
reaches. [Ref. 8] 
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b. Disadvantages 


(1) Expert systems cannot learn from experience, except 
in trivial ways. 


(2) They cannot deal with analogies, something humans do 
constantly. 
(3) They cannot reason intuitively, which is what makes 


human experts approach the level of artistry. 

(4) Expert systems know only the rules. Experts know the 
rules of their domain, but they also know when to 
break them. The computer cannot exercise the 
imagination required to think past the facts and 
rules in its possession. [Ref. 6:pp. 41-44] 

(5) It is difficult to ensure knowledge sufficiency for 
the planned system. The real power of expert systems 
is in its knowledge base; with rapid changes, 
fragmentation, and diversity in the knowledge of the 
knowledge base, the systems! capabilities can be 
severely hampered. 

(6) It is difficult to find an expert or experts who have 
the time to commit to such a formidable effort that 
is required to build a knowledge base. [Ref. 4] 

(7) Application areas have not yet demonstrated 
high returns. 

ley THE sMOpEAs 

An excellent description of the model used in this 
thesis can be found in LCDR Joyce Zellweger's masters thesis 
at the Naval Postgraduate School. This thesis is a 
follow-on study on the selection of successful recruiters 
using a Similar model as adopted in Zellweger. [Ref. 2:pp. 
76-91] 

Past research has successfully identified attributes 


that successful recruiters possess. However, nothing has 


revealed the relative importance of these attributes in 
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relation to each other. This has been a prime concern among 
researchers and is the basis for choosing EXPERT87 as the 
expert system in which to conduct this analysis. EXPERT87 
is a special type of expert system based upon a concept its 
designer has labeled Quasi-Artificial Intelligence (QAI). 
QAI differs from traditional AI in that it is not intended 
for the breadth of problems AI systems attempt to solve. 
Rather, it 1s intended for the large class of moderately 
difficult and repetitive decision problems which so often 
face managers and decision makers. QAI enables coherent and 
objective decisions to be made when no known criterion or 
dependent variable is available for the development of an 
empirical model. It allows efficient interaction of experts 
with a knowledge base, and a presentation of the process and 
results in a form which can be understood by the expert or 
any other user of the system. 

EXPERT87 is not a Simple "weight and rate" process, 
however. It is not decision analysis, nor is it a 
cumbersome and time-consuming AI system with limited 
applicability. Instead, it provides a format for gathering 
intuitive knowledge from experts in minimal time, and in a 
manner that permits verifiable estimation of the trustwor- 
thiness of the expert systems that emerge. The method is 
based on mathematical theory that allows the computer 
program to generate hierarchically ordered profiles of 


hypothetical alternatives (in this case, recruiters). 
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Structuring of a problem's concepts in this way avoids 
cognitive overload of experts and assures a more beneficial 
utilization of attribute information. The software 
generates attribute values for each profile or alternative 
which optimizes the likelihood that the expert's resulting 
model correctly represents the expert's intuitive knowledge. 

Several important principles underlie the development 
of EXPERT87. These include [Ref. 14:pp. 7-18]: 


1. Intuition is a component of thought processes. One 
of the basic principles of decision making is that 
people have sound, intuitive bases for acting on 
given problems, even though they seldom objectively 
express their knowledge. The intuitive knowledge 
they possess is made up of their observations about 
specific attributes of a problem, which they seldom 
explicate when they make subjective judgments or 
evaluations. 


2. Cognitive abilities are limited. In a well-known 
research article of the 1950's, "The Magic Number 
Seven, Plus or Minus Two," psychologist G.A. Miller 
wrote that people cannot effectively deal with more 
than seven concepts at a time. He demonstrated the 
validity of this principle, to within one or two 
categories, across a wide spectrum of human 
perceptual and cognitive activities. For this 
reason, EXPERT87 limits problems to seven concepts 
with seven attributes per concept. Overloading 
human information processing ability is counter- 
productive. 


3. Cognitions are not easily communicated. People cannot 
clearly communicate their thought processes. They do 
not know what information is routinely ignored or 
discounted in their thinking, nor do they know how 
much importance they attach to each concept when 
making decisions. When asked to assign weights to the 
factors which influence their decisions, they are 
often hesitant and sometimes unable to do so. 


4. There is a mathematics of intuitive processes. The 
system is designed to overcome the difficulties 
described above and attempts to capture experts' 
intuitive knowledge without forcing them to think like 
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mathematicians. The interactive process between 
expert and computer generates the functionality 
between attributes of alternatives and the 

overall merit of the alternatives. The functionality 
of the system makes it unique, an expert system which 
is not "rule-based" in the usual sense, but is 
function-based in the sense of being able to express 
in simple algebraic form the fundamental nature of the 
expert's intuitive processes. The system responds to 
each new decision problem using a functional model of 
expert intuition which accurately reflects only the 
consistent and reliable components of these 

Pell clons . 


Hierarchies express relations between concepts and 
attributes. Hierarchies, in EXPERT87, are tree 
structures carefully designed to define a problem in a 
comprehensive, meaningful, and well-organized way. 
Solutions are represented as alternatives, and the 
expert's job is to evaluate the alternatives. When 
the program evaluates an alternative, it calibrates 
the value of that alternative in terms of the expert's 
own set of primary defining concepts 

and attributes. According to L. L. Whyte, hierarchies 
are the most powerful method of classification used by 
the human brain in ordering experience, observations, 
entities and information. 


Hypothetical constructs can be mapped into intervening 
Variables. The difference between concepts and 
attributes is essential to understanding EXPERT87. 
Attributes are specific, people agree on their 
meaning, and people are relatively reliable and 
consistent when rating attributes. Concepts are 
general and people tend to impose their own unique 
meaning on the concepts they use. As a result, 
people usually argue about choices among alternatives 
because they fail to understand that while other 
persons may be using the same word to refer toa 
concept, the concept is not identical to their own. 
Psychologists usually use the terms hypothetical 
construct and intervening variable to clarify the 
distinction between ungquantified ideas and 
operationally defined measures. Figure 2 depicts an 
individual's construct (labeled "V") as a somewhat 
vague and incompletely specified set of ideas. The 
figure illustrates the explication of the construct, 
first by defining it in terms of a set of measurable 
attributes, and then in the generation of an expert 
system. In EXPERT87, expert systems are functional 
definitions of constructs and contain processes for 
mapping information from measurable attributes into 


bee, 





Hypothetical Construct 






eye 


Intervening 
Variable V 


Expert System 
V (XY eee!) 


Definitional 
Process 


Measurable Measurable Measurable 
Attribute Attribute Attribute 
aL 2 3 





Figure 2. Transforming Concepts/Constructs into 


Variables (Ref. 14:p. 81] 


measures, which are labelled intervening variables. 
Then, it can be said that the measure, V, iS an 
operational definition of the construct. Figure 3 
illustrates the substitution of hypothetical 
constructs as concepts of a hierarchy. The problem is 
that the linkage between attributes and constructs is 
missing.  EXPERT87 constructs an expert system to 
provide this linkage after interacting with an 

expert and utilizing its newly-discovered knowledge 
about the way the expert responds to problems. For 
completeness, each concept is linked with its set of 
Subordinate attributes. The expert system must also 
map the derived values of the concepts into an over- 
all assessment of every alternative of interest. This 
representation is displayed in Figure 4. The bottom 
level attributes of communication skills are assumed 
to be eaSily observed and rated. The remaining levels 
represent indirect measurement processes, governed by 
expert systems. | 


Effective decision aides develop understanding and 
confidence. Cognitive psychologists developed the 
underlying principles of EXPERT87 over a lengthy 
period of time. These principles enable the user to 
implement the formal representation of decision 
processes on a microcomputer, to make explicit the 
intuitive knowledge and expertise of decision-makers, 
and to structure decision problems in hierarchical 
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[Ref. 14:p. 82] 


form. The user is then able to define terms and 
concepts and to clarify his/her thinking; 
to develop a clearer understanding of the consistent 
and reliable components of the expert's intuitive 
reactions to sets of information. Furthermore, once 
an expert system is constructed, it is there to guide 
the evaluation of every new alternative that comes 
along in the future, to evaluate thinking about an 
alternative, and to help choose among alternatives. 
The analytic routines EXPERT87 utilizes identifies 
consistencies in the decision maker's subjective judgments, 
and expresses these consistencies in terms of the expert's 
concepts and values. The system then characterizes the 
user's intuitive processes in easy to understand algebraic 
and/or numeric form. 


There are technical limitations to EXPERT87, and as in 


all commercially available expert systems software, it is 
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Figure 4. Schematic of Expert System Development at 
Two Levels of a Hierarchy (Ref. 14:p. 16] 


proprietary. Therefore, detailed information about its 
algorithms and operations is limited. 
In summary, EXPERT87 was chosen for this thesis 
because: 
- the software is easy to use and understand 


- an expert system is developed easily and quickly, 
feedback is immediate, and results are easy to interpret 


- the software can handle a wide variety of decision- 
making problems, and 
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- the program's cognitive engine is deductive rather 

than inductive, which stimulates human thinking and 

reasoning more accurately than the more traditional 

expert systems. 

Figure 5 depicts the specific hierarchy developed to 
model the profile of a successful recruiter. The goal of 
the model is to identify and weight characteristics of the 
successful recruiter, which is the node at the top of the 
hierarchy labelled "Profile of the Successful Recruiter." 

Based on the literature review as well as discussions 
with field recruiters, characteristics believed to be 
related to recruiter success have been identified (Table 2 
in Chapter I). These characteristics are related, and can 
be organized into logical categories. These are 
Communications Skills, Demographic Characteristics, Military 
Background, Personality Characteristics, Behavior 
Characteristics, and Specific Experience. These dimensions 
become the largest branches, or nodes, of the hierarchy. 
The characteristics, or attributes, within each dimension 
become the smaller nodes of the hierarchy. A description of 
the attributes within each dimension are characterized 
below: 

1. Communication Skills 
- Public Speaking Skills 
The ability of a recruiter to stand before a group 
of people and convey information so as to motivate 


an audience is thought to be an attribute a success 
recruiter possesses. 
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Writing Skills 


Although a recruiter's job involves very little 
writing, writing 1S such a large part of 
communicating, that it was included in the model. 


Listening Skills 


Many of the recruiters who tested this model 
believe that listening skills are the most 
important aspect of a recruiter's communication. 

By asking open-ended questions and carefully 
listening to an applicant, the successful recruiter 
can provide information targeted specifically at 
the needs and desires identified by the individual. 


iDgue ene nears 

The successful recruiter has the ability to recall 
information necessary to effectively inform the 
applicant on all aspects of military life. 


Persuading 


tne smecessrul recrurteremust be ablle to close the 
sale. 


2. Demographic Characteristics 


Age 


Experience is thought to come with age. However, 
if the recruiter is too old, he/she may not be 
able to relate to a young applicant; if too 
young, the recruiter does not have enough 
experience to help an applicant. 


Family Support 


An aspect of recruiting that affects the 
probability that a recruiter will be successful 
mene issue Of famlly support, particularly of 
the spouse. Recruiting duty often means living 
in areas away from a military community and 
services the family depends upon. Living away 
from military commissaries, exchanges, and 
medical facilities can create or increase 
financial hardship and stress for families. 
Recruiting also involves long hours, weekend 
work, and travel away from home. 
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SIRO AE 


Most of the literature suggests that intelligence 
1s directly related to recruiter success--the 
higher the AFQT scores the recruiter has, the 
better. 


College Experience 


Education and ASVAB scores are often used as 
readily available measures of intelligence. 


3. Military Background 


Pay-grade 


The recruiters who tested this model all felt that 
the most successful recruiters are E-6's. E-5's 
and E-7's are next, and E-8's and E-9's last. 
E-4's and below do not possess the necessary 
experience to be successful and E-8's and E-9's 
tend to intimidate applicants. 


Years of Service (Active) 

With respect to age and years of service, a 
recruiter must have experience in the service and 
be old enough to have some credibility. 

Years of Service (Reserve) 

For reserve recruiters, some experience ina 


reserve unit is necessary for the recruiter to sell 
the candidate on reserve life. 


Personality Characteristics 


Self-Image 


The successful recruiter has a positive self-image 
and outstanding military bearing. One recruiter 
noted that a sloppy looking recruiter can 
immediately turn off an applicant. 


Integrity 


This attribute showed up most often as the most 
important characteristic within this dimension. 
The recruiter who lacks this attribute is also the 
one with the most fraudulent enlistments and is 
almost immediately weeded out of the recruiting 
game. 
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~—eo xt roverted 


The successful recruiter is more concerned for 
others than himself/herself. 


- Sense of Humor 
This may help a recruiter enjoy the job, and may 
help keep him/her on an even keel in a very 
demanding job. 


- People-Oriented 


The successful recruiter enjoys working with 
people. 


Behavior Characteristics 

mummoeliastanter 
A recruiter's job entails doing just about 
everything on his/her own. The recruiter must be 
able to motivate himself/herself to get the job 
done. 


- Commitment 


To be successful, the recruiter must like his/her 
Job and be committed to it. 


= Flexibility 
A successful recruiter must be able to adapt to 
his/her environment and change plans on a moment's 
notice. 

- Attention to Detail 
To be successful, the recruiter should be able to 
plan activities over various time periods. He/she 


must also be organized so as not to forget a single 
detail. 


- Decisiveness 


The successful recruiter must be able to make a 
decision on his/her own. 
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6. Specific Experience 
- Sales Experience 
Civilian sales experience may be a substitute for 
recruiting experience, since recruiters are often 
described as salespeople. 


- Public Speaking Experience 


A person with public speaking experience has an 
advantage over other recruiters. 


- Counselling Experience 


A recruiter with prior counselling experience also 
has an advantage over other recruiters. 


For each of the six dimensions described, EXPERT87 will 
generate a number of hypothetical profiles which each expert 
will evaluate. The software takes the expert through 
evaluations of attributes within each dimension and 
evaluations of the relative importance of dimensions. For 
example, the software will generate a specially constructed 
set of attribute values for each attribute which defines the 
dimension. The larger the number of attributes within the 
dimension, the more profiles the system will generate for 
expert assessment. This is necessary to provide a sample 
size sufficient to allow for a valid model. Each profile is 
presented in graphic form for the expert to examine, reflect 
on, and assess, aS depicted in Figure 6. For each 
dimension, experts use their own knowledge, experience, and 
intuition to evaluate individuals having profiles of 
attributes for that dimension. The assessment is based on 


the scale shown at the bottom of the graph. The expert 
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enters a score from 00 to 99 depending on his/her overall 
evaluation of that individual. This procedure is then 
repeated for each dimension in the model. 

Once the last profile has been evaluated, the software 
completes its mathematical routines and stores functional 
relationships between attributes and dimensions. Now that 
the expert system is in place, it can evaluate real 
alternatives based on each expert's expertise. An 
additional profile is displayed and evaluated based on the 
expert system just created. After the expert enters his/her 
assessment, the system displays its predicted value of that 
expert's assessment. With reasonable care, the expert's 
response should be accurate to within five or Six percent of 
the system's findings. For example, a small mean squared 
error of 5.0 indicates that the expert system can predict 
the expert's assessment to within plus or minus five points 
or 67 percent of the time and plus or minus ten points 99% 
of the time. The mean squared error also identifies 
instances when an expert might have made a gross error in 
entering an evaluation. 

One of the most important evaluation tools contained in 
EXPERT87 is the Fidelity Index. This index indicates how 
successful the program was in developing an expert system 
that correctly models the expert's own intuitions. If 
Fidelity is less than 80 percent, there is a strong 


indication that the expert's evaluations were inconsistent, 


0 





which means that the intuitive or cognitive processes 
underlying the expert's assessments were not used in a 
consistent way. 

Relative weights are also calculated for each expert, 
indicating the relative importance of each attribute or 
dimension. The expert's judgments are said to be a 
monotonic function of the attribute values. That is, as the 
value of an attribute increases, the value of the concept to 
which it belongs either increases or decreases. The 
software also determines (for each expert) the shape of the 
function of each attribute, whether it is positive or 
negative, monotonic or non-monotonic, linear, convex, or 
concave. 

This information provides the expert with a better 
understanding of his/her intuitive processes and personal 
values. The sign of the non-linear component is actually 
the second derivative of the concept under evaluation with 
respect to the attribute. Positive signs indicate U-shaped 
functions, and negative signs indicate functions which rise 
to a peak, accelerating at a decreasing pace, and then 
reversing. So, if an attribute has a relatively large 
linear component, this means that the value of the concept 
increases linearly with the magnitude of the attribute. If 
the relative weights also contain a significant negative 


non-linear component, this implies a leveling off, or a. 


oak 


reversal of this trend for the larger magnitudes of the 
attmiouLe: fRef. 14:pp. 84-85) 

At no time does EXPERT87 ask the expert to indicate how 
important each attribute is. This information is generated 
by the program based on the expert's evaluations of profiles 
of individuals with specific measured quantities of each 
attribute. The fidelity index is then used as an indicator 
of how accurate EXPERT87 simulates the expert. [Ref. 14:p. 


85] 


Cee ee bo Ee APE RLS 

An "expert" 1s a person who, because of training and 
experience, is able to do things the rest of us cannot. An 
expert is not only proficient but also smooth and efficient 
in the actions he/she takes. He/she knows a great many 
things and has tricks and caveats for applying knowledge to 
problems and tasks. An expert is also good at plowing 
through irrelevant information in order to get at basic 
issues and recognizing problems as instances of types which 
are familiar. [{Ref. 15:p. 5] 

In the recruiter selection problem, the experts are 
recruiters who are either doing the job and have succeeded 
themselves, or recruiter instructors who have been 
successful in the field. Experts 1 through 6 are AGR 
recruiters, with experts 1, 2, 4, and 5 being field 
recruiters from Indiana battalions, and experts 3 and 6 are 


recruiter instructors at the recruiter school in 
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Indianapolis. Experts 7 through 16 are active duty 
Beemmitrers, all of which are instructors at the recruiter 
school. 

The next chapter will analyze the similarities and 
differences of each of the expert systems created by the 
sixteen experts. In addition, a composite model using the 
mean scores of the sixteen experts will be analyzed and 
contrasted with the individual models as well as with a 
"patchwork" model (the patchwork model represents the "best" 
of the experts and will be explained in more detail in the 
next chapter). Trends and relative weights among dimensions 
and attributes will also be analyzed to determine if a 
consistent, clearly identifiable profile of a successful 
recruiter emerges. Finally, an evaluation of hypothetical 
recruiter applicants will be examined to compare the 


different experts' ratings of the same applicants. 


a9 


TIT. ANALYSIS AND RESULTS 


A. THE EXPERTS 

The expert system developed for each of the sixteen 
recruiter experts will be evaluated and compared in terms of 
Fidelity, Standards, and Discrimination (as explained in 
detail in the previous chapter). These indices typically 
range in value from 0 to 100 and can be used to interpret 
the worth of the expert system. (It is important to note 
that the version of EXPERT87 used in this thesis did not 
incorporate a normalization process for these indices and 
the values sometimes exceed 100 or go below 0. (Version 4.2 
Will correct this problen.) 

As previously described, the fidelity index measures how 
well the expert system correctly reproduces the intuitive 
judgments of the expert, the standards index measures the 
extent to which the experts maintain high standards on their 
assessments of hypothetical profiles of recruiters, and the 
discrimination index measures the expert's ability to make 
fine distinctions among hypothetical profiles of recruiters. 
An evaluation of the perfect index for each of the three 
concepts must be left to the user of the system. For 
example, if the user wants an expert who's standards are 
extremely high, he/she would choose an expert with a 


standards index above 80. Similarly, the user would choose 
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an expert with a discrimination index above 80 if he/she 
wants someone who is highly discerning. A fidelity index 
closer to 100 is best in any situation, with 80 being 
marginally acceptable and 40 being very inconsistent. 

Appendix B displays, for each of the sixteen expert 
systems, the three indices, the mean squared error, and the 
explained variance in each of the six dimensions and the 
overall model. 

For the sixteen expert systems, the overall model 
Fidelity Index was above 90 in eleven cases and above 83 in 
the remaining five cases. For the remaining dimensions 
(Communication Skills, Demographic Characteristics, Military 
Background, Personality, Behavior, and Specific Experience), 
the Fidelity Index remained above 80 in all but seven cases, 
with the lowest being 69.1 on Expert 14's assessment of the 
Military Background dimension. Other results with Fidelity 
Indices below 80 were: Expert 1, 77.8 on the Specific 
Experience dimension; Expert 4, 75.4 on the Demographic 
Characteristics dimension; Expert 8, 78.3 on the Personality 
Characteristics dimension; Expert 9, 78.1 on the Demographic 
Characteristics dimension and 75.4 on the Personality 
Characteristics dimension; and Expert 14, 79.6 on the 
Behavior Characteristics dimension. Many of these 
dimensions were either unimportant in the eyes of the 


experts or least important in relation to the other 


3S) 


dimensions and therefore perhaps the experts did not 
evaluate them as carefully as the others. 

For the overall model, the expert's Standards Indices 
ranged form 17.8 to 99.3. A regular active duty recruiter 
(RA) Instructor/Guidance Counselor was the most lenient, and 
an AGR Recruiter Instructor had the highest Standards Index. 
The Standards Indices for the rest of the dimensions varied 
for all experts, but those experts whose Standards Indices 
were high for the overall model tended to have higher 
indices than the other experts for the individual dimensions 
as well. 

The experts! Discrimination Indices ranged from 18.1 to 
103.8. Expert 12, the RA Instructor/Guidance counselor had 
the highest Discrimination Index. Expert 1, an AGR field 


recruiter had the lowest. 


Bee DIMENSIONS 

Table 3 (AGR recruiters) and Table 4 (RA recruiters) 
present the relative weights assigned to the model's six 
dimensions by each of the sixteen experts. The weights in 
each column will sum to approximately one and can be 
interpreted as the relative importance of one attribute in 
relation to the others. For example, Expert 1's weight for 
the Demographic Characteristics dimension is .321, which is 
approximately three times as important as the Behavior 
Characteristics dimension which has a relative weight of 


.05. Communication Skills (.305) is approximately three 


56 


€60° 


So) OM 


mols 


ve 


<i ie 


IBC a 


YOVN 


7SO° 


acc. 


G2 Cj 


910° 


960° 


Lf OO 


€60° S'S 0 Gem E GCC. 
960° is) 0,” eC le Sel: 
88T° Sco: 881° SC 
Eva” LS £909 SHO) 
Ser 6 S)(0) Lots Shel 
ODT” LD Cs SiG GC 

S D t G 


Apu] -SeArasoy 
SNOISNUWEGC HO SLHOIGM FAT LVTdd 


€ ATHVL 


S205 


OS. 0m 


O6T” 


690° 


Ce 


GOE* 


“dxoi- OOaS 
TotTAeyag 
*uOoSidd 
punorbyoeg *TItW 
“oezeUD *oulDd 
STTtAS *“wwood 


UOTSUSUTG 
4190dxq 


oy) 


50: 


681° 


aon ae 


810° 


670° 


ewe 


Lage 


Sy 


aye 


6s 


CLO 


COV” 


ol 


ee) 


shen fg 


Lae 


Coa 


oS 


CU GS 


ST 


eS 


HOG 


ve 


680° 


680° 


[tists © 


VI 


S50) 


Sues 


GAC 


Core 


ENON 


lo 4 


et 


SNOISNHWIG HO SLHOITHM FYALTLVTdd 


5 


Sal 


080° 


GoUs 


Te) 


SUT 


Gat 


OG. 


SOE ~ 


So” 


[8T° 


OGL 


O0€E” 


ti 


CRSA AAE 


LO 


Vale 


0910; 


SOT* 


vS0° 


iS” 


OT 


ApuT -9ATIOW 


2 (DL 


Gol, 


oGCe 


aS > 


ag oe 


Per 


See 


LO |b 


LNG © 


08 


870 - 


cove 


0s 


ne 


US 


ie Je 


860° 


SS Ca 


Deb dnp 22 = Cls 


TOTAeYe 


*uoSsisd 


"yOVeCd TTIW 


2 SBS D 
-oulog 


STITHS “wwod 


uOTSUSUTG 


210dxq 


58 


fees ds important as Specific Experience (.065) ina 
successful recruiter. A more detailed data display for each 
expert is contained in Appendix Cc. 

Table 5 (AGR recruiters) and Table 6 (RA recruiters) 
display the expert systems' most important, second most 
important, and least important dimensions along with their 
relative weights. By order of importance, Communication 
Skills, Demographic Characteristics, and Personality 
Characteristics were the most important dimensions to the 
AGR Recruiters. Military Background and Behavior 
Characteristics were relatively less important, and Specific 
Experience was judged least important of the six dimensions. 
The active recruiters judged Communication Skills, 
Personality Characteristics, and Behavior Characteristics as 
the most important dimensions. Military Background, 
Specific Experience, and Demographic Characteristics were 
all significantly less important. Hence, for dimensions, 
the main difference between RA and AGR recruiters was in the 
Demographic dimension. The AGR recruiters felt it was 
important and the RA recruiters felt it was significantly 


less important. 


ec “ATTRESUIES 

The importance of the attributes within each of the six 
dimensions will be discussed in this section. Again, 
further detail for all attributes and dimensions is 


contained in Appendix C. 
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TABLE 6 


MOST/LEAST IMPORTANT DIMENSIONS 


Regular Army 





mm 


Expert 

Most Commun. }Commun. j}Person. | Commun. | Behav. |Commun. | Commun. 
Important <2603 ~437 E206 ~ 546 ~30'D - 448 .297 
Second Most |Person. |Person. |Demogr. | Behav. oe S. 8 ExP Person. 

Important - 250 atau wai) 2 -214 oe 8 ExP 2D 

Least : : Demogr. jCommun. | S.Exp. S. EXD. Person.|Demogr. 
Important 3 -048 ek «021 .020 .080 AO Fs) 


Most Commun. Commun. Commun. Commun. 
Important pee ~242 ~402 ~434 





Second Most Behav. Person. | Demogr. | Person. 
Important .208 one k ey Ae -194 


Least Person. ; é ‘ Demogr. 
Important .074 : : .0498 





61 


1. Communication Skills 

The attributes within the Communication Skills 
dimension are Public Speaking Skills, Writing Skills, 
Listening Skills, Informing, and Persuading. Table 7 
displays the experts' judgments about the attributes of the 
Communications Skills dimension. Three of the six AGR 
Recruiters judged Persuading most important, while two 
thought Listening was most important, and one felt Informing 
was the most important communication skill. Five of the six 
AGR Recruiters felt writing skills was the least important 
attribute in this dimension, and one said public speaking 
skills was least important. 

Similarly, the Active Recruiters judged Persuading 
as most important in seven of the ten cases. Two felt 
listening was most important and one said informing was the 
most important attribute within the Communication Skills 
dimension. The Active Recruiters also felt that Writing 
Skills were least important in half the cases, Public 
Speaking Skills least important in three cases, Persuading 
was least important in one case, and Informing was judged 
least important in one case. The latter two were totally 
contrary to the consensus of opinions among the majority of 
recruiters interviewed. 

2. Personality Characteristics 
The Personality Characteristics dimension includes 


Self-Image, Integrity, Extroverted, Sense of Humor, and 
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TABLE 7 
COMMUNICATIONS SKILLS DIMENSION 
MOST/LEAST IMPORTANT ATTRIBUTES 


Reserves 


i eee ee eee 
Most Inform.{|Listen. |Persuad. : 
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People-Oriented. As shown in Table 8, both the AGR and 
Active Recruiters consistently identified Integrity as the 
most important attribute within the Personality 
Characteristics dimension. In fact, Expert 18 judged 
Integrity to be eight times more important than Sense of 
Humor, which was the least important attribute. Sense of 
Humor and People-Oriented were consistently judged as the 
least important attribute within this dimension. Most 
recruiters felt that a sense of humor is nice to have, but 
not important to success. The author felt it would be 
important to deal with the high stress factor recruiters 
contend with every day. 

3. Behavior Characteristics 

The attributes within this dimension are Self- 

Starter, Commitment, Flexibility, Attention to Detail, and 
Decisiveness. Table 9 reveals the Judgments within the 
Behavior Characteristics dimension. AGR and Active 
Recruiters again agree on their judgments of Self-starter 
and Commitment as the number one attribute within this 
dimension. The least important attribute has more 
variation, but Decisiveness and Flexibility appear most 
often. The wide variations in these results is probably due 
to the expert's opinions that all of these attributes have 


fairly uniform importance. 
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TABLE 8 


PERSONALITY CHARACTERISTICS DIMENSION 
MOST/LEAST IMPORTANT ATTRIBUTES 


heserves 


fewert [2 [2 [> |[* |s | «| 
Most Image Integ. jlnceg. Integ: Integ. {Integ. 
Important Saree .608 ~449 ~432 207 A oheRs} 
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TABLE 9 


BEHAVIOR CHARACTERISTICS DIMENSION 
MOST/LEAST IMPORTANT ATTRIBUTES 


Reserves: 


Most Detail j;Commit.|/Starter |Starter |Starter 
Important 433 .402 -328 ge TO, reo. 


Least : Decis. Decis. Decis. 
Important : LO .041 gigs 





Regular Army 











Most 
Important 


CC 
Starter |Flex. Starter Detail Starter |Starter 
—29 roo -345 .569 23 O38 POM 
Decis. Decis. Flex. Starter Detail Decis. 
sag .060 .034 O38 pele 035 


Least 
Important 






Most 


Important 


Least 
Important 





66 


4. Military Background 
Military Background attributes include Paygrade, 
Years of Service (active) and Years of Service (reserve). 
Among these attributes, Paygrade was most important to AGR 
Recruiters and Years of Service (Active) was least 
important. Active Recruiters judged Paygrade and Years of 
Service (Active) to be most important and Years of Service 
(Reserve) least important. These results are not 
Surprising. All three attributes measure experience in the 
military which is considered very important when trying to 
sell the service to a potential recruit. Reserve recruiters 
need experience in the Army reserve to sell the reserves to 
recruits, and Active recruiters need experience in the 
active duty Army to sell it to potential recruits. Table 10 
summarizes these judgments. 
5. Demographic Characteristics 
The attributes within the Demographic Characteris- 
tics dimension are Age, Family Support, AFQT, and College 
Experience. AFQT and Family Support were consistently 
judged as most important by both AGR and Active Recruiters. 
Almost everyone said that Age and College Experience did not 
matter. The results show this in Table 11. 
6. Specific Experience 
Specific Experience includes Sales Experience, 
Public Speaking Experience, and Counselling Experience. 


This dimension revealed significant variation between all 
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TABLE 10 
MILITARY BACKGROUND DIMENSION 
MOST/LEAST IMPORTANT ATTRIBUTES 


Reserves 


Expert Mean 
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TREES i 


DEMOGRAPHIC CHARACTERISTICS DIMENSION 
MOST/LEAST IMPORTANT ATTRIBUTES 


Reserves 
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three attributes as to the order of importance. The 
recruiters interviewed felt that any of the three attributes 
help, but very few recruiters have any kind of these 


experiences. Table 12 displays the results. 


D. COMPOSITE MebEES 

In order to obtain composite models, two different 
methodologies were used. 

1. Mean 

Two separate mean models were developed, one 
for reserves (MAGR) and one for active duty (MRA). To 
develop this model, all assessments from each expert were 
sorted by concept and response, and the means of the 
responses were calculated. The means were then entered into 
the expert system to create a composite expert. 

For both RA and AGR Mean expert systems, the overall 
model Fidelity Index was 97, and was at least 96 for the 
individual dimensions. The Standards Index for the overall 
model was 60 for the reserves, with the individual 
dimensions ranging from 48.3 to 67.5. The actives Standards 
Index was 74 for the overall model, and ranged from 46.3 to 
74 in the individual dimensions. Finally, the Discrimina- 
tion Index for the reserves was 62 for the overall model, 
and ranged from 53 to 68 in the individual dimensions. The 
actives were less discriminatory with an index of 41 for the 
overall model, and a range of 38 to 65 in the individual 


dimensions. As expected the use of mean input values 
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TABLE 12 


BERCIPIC EAPERIENCE SDIMENSTION 
MOST/LEAST IMPORTANT ATTRIBUTES 


Reserves 
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stabilize the disparities between individual experts by 
creating an "average" expert system. The mean expert is 
included in Tables 3 through 11 for comparison and summary 
purposes. 
2. Patchwork Models 

EXPERT87 contains a feature which allows the user to 
create a composite model using the experts who are logged 
onto the system. The user can patch experts to concepts or 
dimensions based on any arbitrary criteria. I have defined 
the criteria using a high fidelity index (as close to 100 as 
possible), a normal (around 50) Standards Index, and a 
normal (around 50) Discrimination Index. There is no 
"ideal" criteria. The actual decision should be made by the 
user of the system. However, for purposes of analysis and 
based on my judgments, I have defined the criteria as 
stated. For example, Expert 1 meets the criteria for 
Communication Skills. That is, high Fidelity, normal 
Standards, and normal Discrimination Indices. However, 
Expert 1's assessment on Personality Characteristics is 
below minimum. But, Expert 2 meets the criteria for 
Personality Characteristics. So, the system allows you to 
patch the expert with the concept and select recruiters 


based on this composite model. 


Eve COMPARING THE EXPERT SYSTEMS 
This phase of the analysis used the eight AGR expert 


systems (six experts, the mean model, and the patchwork 


v2 


model), and the twelve RA expert systems to evaluate a set 
of twenty hypothetical recruiter applicants. Subjective 
assessments were made to determine measures on the 
attributes, trying to make them as realistic as possible. 
Many of the attributes in the model cannot be measured 
objectively, however, so an assumption was made that the 
ratings (1) were measurable and (2) were agreed upon by the 
experts whose systems were used to evaluate the 
"applicants." In other words, the experts all assessed the 
same twenty applicants as presented by each applicant's 
profile of attributes. The experts will be compared by 
their relative evaluations of the hypothetical recruiter 
candidates. 

Table 13 displays the profiles of the twenty 
hypothetical recruiter candidates. The profiles were 
designed such that some of the applicants are at the top end 
of the rating scale (0-99) and some at the bottom on all 
attributes. These cases will illustrate how judgments are 
affected by the Standards Index. Experts who have high 
standards tend to assign lower ratings than more lenient 
experts. 

The remainder of the recruiter applicants also have 
specially constructed sets of attributes. All of the 
applicants meet the minimum requirements set by the Army: 


1. At least a high school diploma graduate or GED with 
one year of college. 


2. Minimum GT score of 110 waiverable to 100. 


73 


TABLE 13 


ATTRIBUTE RATINGS OF TWENTY HYPOTHETICAL RECRUITER APPLICANTS 
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TABLE 13 (CONT. ) 


ATTRIBUTE RATINGS OF TWENTY HYPOTHETICAL RECRUITER APPLICANTS 
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3. Between 21 and 35 years of age. 


4. In paygrades E-5, E=6, 868 E-7 (BE-7 "Soma snaveune mien. 
than 2 years time in grade at time of selection). 


The minimally acceptable rating on the scale used by the 
software is 45. The resulting rankings of candidates is 
contained in detail in Appendix D. 

Tables 14 and 15 presents the results of the expert 
systems evaluation of the hypothetical candidates. 
Asterisks indicate the rejected applicants (below 45). As 
expected, the results are very similar between the AGR 
expert systems and the Active expert systems. In almost 
every case, the top five applicants are B, L, E, C, and O. 
Every case rejects A, K, and N and they show up as the last 
three applicants. Applicant B was assessed at the upper end 
of the rating scale (all 9's). The overall profile scores 
for Applicant B range from 41 to 74.3 in the AGR assess- 
ments, and from 41 to 83.1 in the Active assessments. The 
low rating makes those particular expert systems suspect 
because they are so contrary to the majority opinions. It 
also makes no sense to reject an applicant who is superior 
in every dimension as candidate B is. A review of Experts 
12 and 14's indices in Appendix C show their expert systems 
to be at extremes in the Standards and Discrimination 
Indices. The Mean expert for AGR recruiters also rejects 
applicant B with a score of 41 however. This is surprising 
but may be explained by the relatively low ratings overall 


given by the AGR recruiter experts. Results also show that 
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TABLE 14 


EXPERT SYSTEMS EVALUATE HYPOTHETICAL APPLICANTS 
Reserves 
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EXPERT SYSTEMS EVALUATE HYPOTHETICAL APPLICANTS 
Re cUeecar 
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the expert systems created by Experts 5, MAGR, 9, and 14 had 
the highest standards and rejected more than half of the 
applicants. 

Some interesting results are in the comparisons of MAGR 
and the patchwork model (PAGR), and MRA and the patchwork 
model (PRA). The mean and patchwork models for AGR 
recruiter experts are quite different. Although both models 
select Applicant E for example, MAGR ranks E as number one, 
while PAGR ranks E number seven. However, MAGR's rating is 
60 as opposed to PAGR1's rating of 62.8. This is a good 
example of MAGR's high standards. Also MAGR rejects 
thirteen of the twenty applicants while PAGR rejects only 
1c ONe has 

In contrast, MRA and PRA are surprisingly similar. They 
both rank B and L first and second, respectively. Similar- 
ly, MRA and PRA reject applicants U, F, G, K, N, and A in 
exactly the same order with similar ratings. PRA also 


rejects H, T, and D however, and MRA selects them. 
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IV. CONCLUSIONS AND RECOMMENDATIONS 


A. CONCLUSIONS 
1. Past Research 

The objectives of this thesis were to identify the 
most important factors and their relative importance for 
identifying individuals with high likelihood of being 
successful recruiters. 

An extensive literature review was conducted ina 
prior thesis conducted by LCDR Joyce Zellweger in December 
1986 at the Naval Postgraduate School, Monterey, California. 
It identified many previous studies that examined the 
recruiter selection problem. Two distinct types of factors 
have been reviewed for their utility in predicting 
successful recruiter performance. One class of factors is 
focused on biographical and personal history variables which 
can be found in standard military personnel files. These 
factors are popular because they are easy to identify and 
measure. The biographical and personal history variables 
most frequently used were age, education, entrance test 
scores, gender, marital status, etc. However, these 
variables are extremely diverse and probably differ in terms 
of the underlying dimensions they are intended to reflect. 
Knapp and Benedict's 1986 study found that the predictive 


validity of individual variables contained within this 
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category can differ significantly as a consequence of 
predictors used and the type of criterion being predicted. 
Furthermore, the vast majority of previous studies focusing 
on a small number of specific biographical or background 
variables were found to be limited in their effectiveness 
when used alone to predict sales success. PREtieeo. pp. 19, 
22.3 

The other class of factors are given by specific 
tests to measure personality and behavioral characteristics. 
Both sets of factors yielded disappointing results. 
Recruiter characteristics found to be significant varied 
across studies and few results were cross-validated or could 
not be duplicated upon cross-validation. 

However, ina 1986 study by Russell and Borman, 
researchers found that vocational interest and personality 
variables (such as dominance, self-confidence, and 
spontaneity) were significantly associated with military 
recruiter performance. Further, cognitive variables such as 
verbal ability and general aptitude appeared to have little 
validity for predicting military recruiter success. They 
also found that although military recruiting is essentially 
a sales job, the type of "product" sold is quite different 
from sales jobs in the civilian sector. Recruiters sell 
careers or life-styles, not material goods or services. 
Russell and Borman's study also researched civilian sector 


sales jobs and found that even though "product" type 
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differences exist between the military and civilian sectors, 
there are major consistencies between them which contribute 
to military recruiter and civilian sales success. For 
example, personality variables such as dominance have 
resulted in validities in both arenas. Also, aptitude and 
verbal ability measures have shown little merit for 
predicting either recruiter performance or civilian sales 
success. Skill level variables, such as assessment center 
scores, may be useful predictors of recruiter job 
performance. [Ref. 17:pp. vi-vii) 

In most prior studies, the criterion problem was 
probably the single most reason why relatively little 
variance in recruiter productivity was explained. The poor 
findings were usually a result of the failure to properly 
conceptualize predictor-criterion relationships. Objective 
criterion measures usually reflect the results of 
salesperson behavior and environmental factors. [Ref. 16:p. 
33] 

2. Expert Systems 

This thesis applied a relatively new methodology to 
the recruiter selection problem--Expert Systems. Expert 
Systems is a field of Artificial Intelligence which has been 
particularly successful in solving the types of problems 
where there iS incomplete knowledge and where judgment is 


needed. 
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The expert system shell utilized in this thesis is 
called EXPERT87. It is a special type of expert system 
intended for the large class of moderately difficult and 
repetitive decision problems which so often face managers. 
It allows efficient interaction of experts with a knowledge 
base and develops a model using one of more experts' 
knowledge, judgments, experience, and intuition to solve 
problems. 

EXPERT87 also addresses the problem that studies 
using personality and behavioral characteristics have not 
been able to address of revealing the relative importance of 
successful recruiter attributes in relation to each other. 
Since it does not rely on an objectively measured criterion, 
this approach also avoids the problem of poorly specified 
and measured performance criteria that has plagued much of 
the previous empirical modeling efforts to profile 
successful recruiters. 

3. Profile of the Successful Recruiter 

The model developed for this thesis was based on 
characteristics previous studies found to be related to 
recruiter success as developed in LCDR Zellweger's thesis, 
and then refined to obtain a more efficient set of 
attributes. These attributes were organized into larger 
dimensions of the hierarchy. 

Six Army reserve and ten active duty Army recruiters 


evaluated the model, and EXPERT87 estimated their expert 


83 


systems. In addition, I developed two composite experts 
using the means of all assessments made by the experts. 
This allowed me to develop a composite model and compare 
Similarities and differences with the individual models. I 
also used a patchwork model (which is part of the EXPERT87 
software) to develop alternative composite experts. The 
results were very similar as to which characteristics were 
more important than others. The AGR recruiters felt that 
Communication Skills, Demographic Characteristics, and 
Personality Characteristics were most important in a 
successful recruiter. The active recruiters felt that 
Communication Skills, Personality Characteristics, and 
Behavior Characteristics were most important. Within these 
dimensions, the most important attributes were Integrity, 


Listening, Informing, Persuading, AFQT, Family Support, 


peed) 


self-starter, and Extroverted. The mean models also support 


these results. 


B. RECOMMENDATIONS 
1. Measure Important Attributes 

Personality and Behavior characteristics come up 
time and time again as one of the top concepts related to 
successful recruiters. Described below are a variety of 
measurement instruments designed to measure personality. 

a. Myers-Briggs Type Indicator (MBTI) 

Isabel Briggs Myers and Katharine C. Briggs 


developed this test which consists of four dichotomous 


84 


= —— loo SC 


indices of personality types: Extraversion-Introversion 
(EI), whether perception and judgment are directed toward 
the environment or the world of ideas; Sensation-Intuition 
(SN), indicating dominant perceptual style; Thinking-Feeling 
(TF), which one of these two modes of judgment is relied 
upon; and Judgment-Perception (JP), indicating which of 
these is relied upon in dealing with the environment. The 
test consists of 166 forced-choice (usually two) items. 
Fifty-two items are word pairs in which respondents indicate 
a preference. Some of the pairs are theory-certainty, 
build-invent, casual-correct, who-what, sign-symbol or 
Similar to the following: 

Do you: 

(1) prefer to do things at the last minute 

(2) find it hard on your nerves 
The test is self-administering and has no time limit, but 
usually takes about 50 minutes to complete. The MBTI is 
easy to administer and score, and the types do have the 
virtue of being mutually independent. A draw-back for our 
purposes is that it only measures a couple of the attributes 
identified (extrovert and self-image). It is available 
through the Educational Testing Service, Princeton, New 
Jersey 08540. [Ref. 18:pp. 186-189] 

b. California Psychological Inventory (CPI) 
The CPI was developed by Harrison G. Gough. It 


groups eighteen variables under four classifications: Class 
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I measures poise, ascendancy, and self-assurance; Class II 
measures socialization, maturity, and responsibility; Class 
III measures achievement potential and intellectual 
efficiency; and Class IV measures personal orientation and 
attitudes toward life. This one test measures most of the 
attributes identified in the expert system approach to 
profiling the successful recruiter. Specifically, it 
includes measures of self-starter, extroverted, people- 
oriented, self-image, flexibility, commitment, and 
indirectly, integrity. Integrity could be measured using 
the variables, responsibility and socialization. They are 
defined by the CPI as follows: 

(1) responsibility--indicating seriousness of 
thought and manner, conscientiousness, dependability 
and uprightness; being the kind of person that others 
tend to trust and to rely upon. 

(2) socialization--indicating a strong sense of probity 
and propriety; acceptance of rules, proper authority, 
and custom; a person who seldom if ever gets into 
trouble. 

The CPI is essentially self-administering and 
consists of 480 statements. The 18 scales are normative and 
are based on over 6,000 males and 7,000 females. The raw 
scores are converted to profiles which provide graphic 
representations of standard scores. 

Convincing evidence exists to validate each of 
the 18 scales. Even attributes such as self-acceptance 


revealed significant differentiation between high school 


students rated as high and low on self-acceptance by staff 
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assessment ratings. In test-retest reliabilities reported, 
high school subjects were tested after one year. The median 
test-retest correlation was .65 for males and .68 for 
females. Peeks lo ee. 37-40] 

c. The 16PF 

The 16PF is a personality test designed to 
measure an individual's personality in terms of sixteen 
basic factors. It was used successfully in a predictor 
battery in a Marine Corps Study conducted by Larriva (1975). 
Some of the factors measured which have been associated with 
recruiter success were dominance, aggressiveness, self- 
confidence, and spontaneity. 
2. Test the Model 

Before making any further modifications to the 
model, field test it. An appropriate expert system could be 
based on criteria set forth by the Recruiting Command and 
input values measured for the attributes of selected 
recruiter candidates at entry to recruit training school. 
These recruiters could be tracked for at least one year to 
determine their performance and the validity of the model to 
detect potentially unsuccessful recruiters. The results of 
the validity analysis could then be used to modify the 
existing model. 

3. Improve the Model 
Results of this analysis and suggestions from the 


experts who participated in this project indicated that some 


87 


of the attributes in the model may be eliminated without 
affecting recruiter selection. In other words, they are not 
important to recruiter success. The attributes identified 
as unimportant were writing skills, age, sense-of-humor, and 
decisiveness. By giving these attributes a weight of one, 
the model could be tested to see if the results of the 
twenty hypothetical recruiters remains the same. If they 
do, these attributes could be eliminated and probably not 
affect the model. 
4. Work Remaining 

Many possible military personnel selection 
applications exist for this type of methodology. This study 
revealed very few differences between reserve and active 
duty recruiters in their perceptions of what makes a 
recruiter successful. This same model could be applied to 
the active Army in selecting recruiters and could be tested 
across the services to determine whether there are 
Significant differences in their perceptions and knowledge 
of what characteristics define a successful recruiter. This 
methodology could be applied to any type of personnel 
selection problem where consistent and objective decisions 
must be made and objective performance criteria are 


difficult to measure. 
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APPENDIX A 


ANNOTATED BIBLIOGRAPHY 


Department or the Arniy Vauly 1933) 

Noemy saecrhat ing: Conmand 

To prescribe the procedures, criteria, and 
petsconnme!l aCemens required for the selection and 
AeomcimMemerOor Keoullar SAriny and Reserve enlisted 
Poe Niel On yoChY heCrds wo, ARMY recruiters; 
FE Outlines the policy concerning selection and 
assignment of enlisted personnel to USAREC 
administrative support positions; and it 
prescribes the management policies applicable to 


all enlisted personnel while assigned to USAREC. 
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Method: 
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Borman, W. C., Toguam, J. L:, and eos pe wna. 
(May 1979) 

An inventory Battery to Predict Navy andeiatige 
Corps Recrulter Performance: Developmenmmuana 
Validation 

To develop peneli—and-pen predlvetors Gieia, 
and Marine Corps recruiter effectiveness and 
evaluate the validity of these measures. 

A trial predictor battery of personality, 
interest, and biographical items and scales 

was developed based ona literature review of 
previous military recruiter selection studies. 
The battery was given to a geographically 
representative sample Obese  wiavy and lie 
Marine Corps recruiters. Scores on the 
predictor measures and performance ratings were 
correlated. An objective effectiveness index 
was also used in a concurrent validation 
design. The relationships between performance 
criteria and the various predictors were 
assessed. 

a. In the Navy sample, the estimated cross- 
validities for predictor composites against 
four of the five performance criteria were 
Significantly ditterent Erom Zero at tne Fun 
bevel (Ranged from 217s toes. 

b. In the Marine Corps sample, validity 
estimates ranged from .22 to .38, with all five 
predictor composite -pertoemance criterion 
relationships significantly dlfferent from zera 
at the .05 level. 

c. This predictor battery shows promise for 
helping the Navy and Marine Corps decision- 
makers in selecting recruiters. 

a. The researchers went to great lengths to 
arrive final performance criteria that 
reflected relevant, reliable, and comprehensive 
measures of recruiter performance. AS a 
result, reasonably good validities were 
obtained in the study. 

b. Controlled tonmthe tacveenatesubjeccrive 
performance evaluations are sometimes unreli- 
able by paying careful attention to defining 
performance dimensions and selecting the proper 
persons to provide the ratings. 
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Que miereross=-Validity CoeEbreients computed 
are only estimates v£ Lhe composites! 
predictive validity. 

Dee Pues mero trative sec fOr persons in Lhis 
study are probably different than the kind of 
set they would have had if they were arftually 
Pie ewes hive neOtitcai Ss applicants wwetore 
Pelt Womeecepeed for Trecrmiting duty. 

ete oer tors ut leniency, restriction of 
Pie gana ta LOMarenevalvacred only indirectly. 
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Elig, T. W., Gade, PB. A., and Jounsonmeaeee 
(undated) 

Recruiter and Recruit Demographic 
Characteristics Ay ete liminary eave see 
tion Of RECrul Ceresere cel oOnmer lt ecmla 

To describe a mew approach to recruiter 
Selection research. 

Utiliges the MiPitary Enlistmene Processing 
Station Reporting System (MRS) Eo develop 
measures of recruiter performance based on 
YreCruiUumeharaclteristi¢Gs. Cnew source Oleic 


4 





used was the Army's Enlisted Master File (EMF). 


A sample of 612 was selected based on three 
concerns: 4. Toe@idlent ity soroduer: oO) 
recruiters From wonm-product ton personnes, 

Db. The recruiters must have sufficient time 
as @ Eull production reerurrer ac the came 
location Eo help Gontrol Miouwopposruntt as! a4— 
and seasonality fluctuations, and c. The 
Sample ShoOUuIG GOonCaAlnita cut fremene There ot 
females to do cross gender comparisons. 

The analysis falls into two categories: 

a. Demographic and background characteristics 
of recruiters were systematically explored 
througn descriptive statistics. 

b. Recruiter personnel characteristics were 
Systematically ¢orrelabredywwehsecomenace 
production and identified through analysis of 
covariance techniques. 

This study demonstrates that recruiter demo- 
graphic characteristics can be related to 
recruit characteristics when opportunity bias 
is removed. The best Army recruiters were 
found to be better educated, have a higher 
AFQT, and if male, younger, and if female, 
Older. Thus, 1t appears that demographic‘data 
may be useful for selecting recruiters from a 
non-voluntaxry see. 

Controlled for opportunity bias=and Identitica 
some interesting attributes of successful 
recmuiee nS: 

The small size of the sample resulted in many 
variables not able to be cross~validated to 
test for interaction etreeccmuparceleu lar, 
gender). There were only 60 women recruiters 
in the data base. 

The number of applicants contracted in FY79. 
adjusted by the covariate, DRCAVG (opportunity 
Dias Jt 

gender, education and AFQT (renormed), 
recruiter's age, recruiter's ethnic group 
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DEOWiewee HS WOOUd, M.D 
(May 1978) 

pai ecm werss) Colhermon Developmene arid 
Pocuimeneay. Va lbiation site oc lection 
Procedure 

ReeierveTOpea Valid™-criterian of recruiter 
siftectiveness and develop a test battery to 
identify Chose most life@ly te succeed as 
recruiters. 

Stepwise Multiple regression; Benchmark 
Achievement Scores (BAS); Simple Achieve- 
Heme ceores (SAS). 
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BAS Score = (Actual Production/Predicted 
Preduction) x 100 
BAS Scure -= (Tetal Production/Average 
PeOci@edtonmemy. OC jeex, 100 
a. SUbBOE Che varMe@nce in production scores 
Mwerives from factors unrelated to individual 
Peeriniter = Characteristics. 


b. Simple Achievement Scores (SAS) appear to 
be a more equitable measure of a recruiters 

St recelveness Chan wetier more Craditianal 
measures. 

Cc. Twenty background items that may be useful 
were identified for selecting recruiters. 
Peeler r leon icemineormation does not refer to 
Bie vepee Wi lessjecogrammledal area Wm a recruiter's 
area because the information is not available. 
b. Unemployment rate, median family income, 
and education level of community came from the 
1§¥.S. Census Bureau and dala is organized only 
Oye ey eer Chey -ianyebeerultcing territories 
Comprise portions o£ several courties. 

Sec emMinliouerlimg 6Me Leste —saltery--— 
administered at two locations, one a busy 
recruiting station and the other, a group 
testing situation was not achieved. The 
ultimate effect could not be ascertained. 
Total Production 

Average production per recruiter in subject's 
Pisteric: Reecruitimegmeonmand {PRC}; Average 
market share for station zone; proportion of 
Zener subpurtban  meomuno OL reerUulter experience; 
MuMmBe ENOL i gieseneaol Seniors Im Zone; average 
Pecdternonmpe@ Beerurcewmror Subject'’s Regional 
Recruiting Command (RRC); number of ASVAB's in 
Subjeece'S PRC; mumber Gf 17-21 year olds in 
Semrege listation zone; size of station zone 
iiesalare Miles sD Loporkion Of Zone rural; 
DEODOTE1 OM Or eZzene metro, ratio of qualified 
military available (QMA) to military available 
CAD) 2 
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Tit be: 
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Method: 
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Borman, W.C., Hough, LsMee and BPinne eearee i.) 
(Feb yell 76 } 

Development of Behaviorally Based Rating Scales 
for Evaluating’ the Permiormances on Uo mela 
R€Grumte ns 

To develop and ftieid test performance rating 
scales for measuring Navy recruiter job 
effectiveness - first phase of research. 

Used behavior sc€aliigmnethodetody t€ogeamaes 

800 critical “incidents deserrping di teememe 
Facets of effective and ineffective recruiter 
performance from field recruiters, their 
Superiors, and recruits. These incidents were 
classified into nine dimensions: a. Locating 
and contacting qualified prospects, b. Gaining 
and maintaining rapport, CC. Gbtaining intozma. 
tion from prospects and making good person-Navy 
fits, d. Salesmanship skills, e. Establishing 
good relationships in the community, ft. Provide 
ing Knowledgeable and accurate information 
about the Navy, g. Administrative skills, 

h. Supporting other recruiters and the command, 
and 1. Dedication to Ghe Job. 

In addition, a different group of recruiters 
made similarity judgements between every 
possible pair of a subset of 60 behavior 
examples randomly chosen from a large pool of 
incidents. The results were analyzed using 
nonmetric multidimensional scaling (MDS). 
Regression analysis was used to define the 
pattern of contributions made by various com-— 
binations of the MDS dimensions to each of the 
retranslation dimensions. MDS dimensions were 
a. Gathering information about applicants, 

Db. Planning and organizing recruiting 
practices; looking ahead to future recruiting 
requirements, c. Expending extra etfort to aid 
applicants or recruits, d. Salesmanship; 
listening to the prospect and then making an 
appropriate and effective sales pitch, and 

e. Bxpendimg extra ef Eortsee laweadm- oO. DrospecEes 
Ing aGervy Teles: 

Identified more than 800 critical incidents 
describing different facets of effective and 
ineffective recruiter performance. Results of 
the field test were analyzed and revealed that 
self and peer ratings contained impressive 
convergent and discrimanant validity. 
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Stren: 


Paka: 


Met objective successfully. Good first step in 
research. The rating scales provided an oppor- 
Bee eemdssess Ene Vala lty sot proceaur es 
presently being used and those developed in the 
Pumre torselectmndividuals to recruiting 
duty. Useful to "educate" Commanding Officers 
and persons considering recruiting duty. 
ReSetEtetea ©O Self and @peer ratings to assure 
highest reliability and valid performance 


appraisals. Supervisory ratings are used to 
portray overall job effectiveness of each 
recruiter. Data was insufficient to conclude 


unequivocally that the rating scale-rating 
Source assignments would always be optimal in 
obtaining valid performance indices for 
recruiters. 

The fourteen variables described in the 
"method" section above were used for correla- 
tional studies comparing the two sets of 
dimensions (Behavior scaling and MDS). 
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Borman, W.C., Toquam, J.L., and Resse yar. 
(Mareh 1977) 

Dimensions of the Army Recruiter and Guidance 
Couns 6 orale 

Determining performance requirements of the 
recruiter job by defining underlying task 
dimensions associated with recruiter and 
guidance counse lompsobse 

Multidimensional Scaling (MDS) and a clustering 
procedure 

A composite list of four broad dimensions (Pros- 
pecting activities, Publicizing the Army, 
Selling the Army, and Administrative Activities) 
was formed and could be useful in developing 
selection procedures for potential Army 
recruivems. 

The MDS solution iS a general framework for 
representing the four broad dimensions and the 
clustering solution defines the MDS dimensions 
more specifically. Could be useful in identify- 
ing attributes Necessary seomemecec a minma 
recruiter job. 

Did not identify personal characteristics and 
attributes of successful Army recruiters. 
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—poemones Graham, W.R., Brown, G.H., kKine® W.L.3 sand 


WO OC sete. (March 1979) 

iret ie: Sef Oc oeliany Of Army Reerwiters: Their’ Job 
Behaviors and Personal Characteristics 

OD) : Pilot study to develop hypotheses concerning 


Phe weiatalcterisl ics ande ven behaviors associ- 
abea Wilt PeCrulter success. 

Method: Pieervlewseor 79 recrumiters including subjects 
Aeneid average, and fiowe records of success, 
in terms of percentage of quota achieved. 

‘Giolg en ie eee SaseGadwon SeUE-“description data, few 

Characteristics were significantly related 
ESmeroauect ion reeords . 

b. High producers were less likely than low 
producers to cite "independence" as a 
Suu@mee OL Seb Satistactlon: 


c. High producers were less likely than low 
DEVamGceeseroecomplaim about bong nours of 
WOLrk. 

d. High producers more often than low 


producers mentioned the use of Pre- 
Piet toOnmme mys teatmle te) cards and mail-— 
outs as prospecting techniques they had 
found successful. 


e. High producers less often admitted 
communication problems. 
£. High producers were less likely to describe 
themselves as "not irritable" and "empathe- 
ees a 
Sbren: Gaede FlESteslep fOr —FULEMer research. 
Weak: Successful in meeting its principal objective, 


but findings suggest that local situational 
factors may have such impact as to preclude any 
Simple relationship between selection variables 
ane er leerlon Der rEoOrmance.  omall sample size, 
Domereepreesenudr ive --aie a reeruiters were from 
ioe roe ReCrULcing District. 
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Hirabayashi, DD.) Sand) HersGhyen. (Deere I 85} 
Excellence in Navy Recruiting 

To document characteristics of excellent Navy 
Reeruiting Distrieuse 

Interviews 

Navy recrulters are movers, shakers, and sales- 
men; hungry for success/promotion; aggressive, 
want responsibility and to excel; have outstand- 
ing communication skills, fundamental knowledge 
of recruiting, inherent skill with numbers, 
sales, and the public; ambitious, extroverted, 
and like to meet and talk to people; positive, 
cheerful, enthusiastic, and self-motivated. 
Although not useful for empirical analyis, couta 
be great information for expert systems. 

Did not develop a model - was not tested. This 
study was based only on interviews. 
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Vovackw Le, aand Kipmis; Dei March 1350) 
Development of a Device for Selecting 
Recruiters 

Gemaevelop a Navy recruiter selection 

battery and conduct a concurrent validity 

study to assess the battery's usefulness as 
ANSeCrLeenwenG device. (Navy study) 

Tie predwee tor Latltary was composed of 153 
measures designed to reflect £Eluency of 
expression, knowledge of the Navy, interest 

in recruiting activities, and general aptitude. 
The validaticen sample consisted of 410 active 
duty recruiters representing 40 main vecruiting 
Sete lLoOns sous cabs, ana Dereanen Stat ions. 

The sample was formed by calling the commanding 
officers and asking them to nominate the most 
and least erfective recruiters from their 
respective stations. The nominations were 

used as the criterion measure against which the 
predictors were validated. 

ine sample used to cross-validate the predictor 
battery consisted of 260 students attending the 
6-week recruiter course. 

Willingness to work and recommendation for 
VeerhlLEimg aur y  eOrkhetakeegssragmniticantly {(p < 
Towne Peechateal competence. The only other 
Pecad tc. ors 4iven, cOrrel acea, Significantly with 
aye Ol Chew ermn etiam was er lne ne y-Ot-expression 
weenie  eldiedea Comueltatron.of -~,18 (p , 401) 
with Military manner. Overall, the results 

DE the cross-Validation analysis indicated that 
a successful recruiter haS persuasive 
interests, is not overly interested in 
Senolarly pursitte mana babieves in the value 
of a Navy career. 

tie wrecalerOor bakery eaneaing a Variety of 
tests and inventories with reasonably high 
reliability of the criterion nominations. 

Only a limited number of the scales and item 
RevyewcrOss-Validavead Significantly. This is 
Para y ue EON bio umaeceurnat Ene raters were 
making their evaluations on the basis of 
reputation instead of performance or because 
many OF Ene sindividualwartrrerences that are 
meet eGelVenOt Gee rulter Success were not 
included in the battery. 
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Massey, I. M., and Mullins,” C. i ige ee oc 
Validation Of the Recruiter-Salesman Selection 
Tesi 

To validate the Recruiter-Salesman selection 
test. 

An eight inventory battery was developed to 
measure qualities such as empathy, friendli- 
ness, sociability, and perseverence. Predictor 
Variables were correlated with school success 
and supervisor field ratings. 

Cross-validation results indicated that the 
battery would be useful only marginally in 
predicting sencoel pertormance and not aera) ieee 
predicting flea erating s. 

Supervisor rating criterion probably caused the 
poor results because of “halo" and "leniency" 
eftects:. 
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Dein «3 
Exp ban : 


Kime ora... 197) 2) 


ESvienoleilcaleand Demographic Predictors of 
Success aS a Naval Recruiter 

To develop a personality test which would be 
useful in predicting Sales ability. 

The typical effective Navy recruiter is 
married, has more years of formal education, 
and CenGdeeeambme Warm, Outgoing, dominant, 
aggressive, self-assured, and has relatively 
conservative political views. This battery was 
foe womsemeeci (oe Op hot ore ecermunl ting 
assignments between 193972 and 1976. 

identified possible attributes of a successful 
recruiter; cross-validation was performed and 
indicated the equation would be useful. 

Actual use proved the equation did not predict 
sales ability effectively. 

Success as a Naval Recruiter 

Mami diestadkse mn cabSeo: BdUCation; X3 (score 
mange Ofe0-2) Xt score range of O0-5)3 .X5 
(score range of O-11); Warm/Aloof; Dominant/ 
Submissive; Apprehensive/Self-assured; Experi- 
menting/Conservative; MD (score range of 0-14) 
Cease serimehon DeLtoOLremouash 1S descriptive 
of higher scores, above 5.5; below slash, lower 
Seow eS...) 
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Dep. Va: 


BX nay es 


Shupack, M. Age unes 1979) 
An Analysis Or the Cost Implications Ou uO ee 


SS Gs [VD © <UqGnNWENMGees> | sisiNlciomESSEEEESAISanESaTneon 


SE Select ing Navsaeccruucers 
To identify success predictive criteria for 


selection Of Navyevecrurters using 
NAVCRUITCOM*s Homor Roll aS MOR. AlSO prom em, 
and explain the variance in recruiter 
performance. 
Multiple Regression Analysis on the entire 
Sample group as well as three subgroups: 
ALLMEDIOCRE - personnel who completed the 
entire 20-month test persed: 

ALLSUCCESS - personnel who completed the test 

period and achieved the MOE. 
ALLDROP - those who failed to complete test 

per mod: 
Test period = Jan. 7 130) eatug. 30, 
Sample - 1262 male, active duty canvasser 
recruiters. 

A. The characteristics which contributed most 
to the explanation of variance for the success- 
ful recruiters (ALLSUCCESS) were education, 
paygrade, and enlisted entrance test scores. 
For unsuccessful recruiters (ALLDROP), the best 
predictors were rate, active duty, and enlisted 
CENELANCe Bee o roe ones. 
B. Successful recruiters were most often in 
paygrades E5, E6, had 6-14 years of active 
duty service, GAM scores between 130-150, SAM 
scores between 65-30, and High School diploma 
or up to two years of college. No strong 
trends regarding rate were distinguishable. 
C. Recruiter attrition has substantial finan- 
Cial implications. 245 personnel were in the 
ALLDROP group of this study, costing almost 
three million dollars in base pay, BAQ, and PCS 
costs alone. 
Reasonably good identification of factors which 
are indicative of success and failure. 
None of the statistical analysis identified 
Characteristics of the mediocre group 
(ALLMEDIOCRE); all values for all variables for 
all individuals in the sample were not 
obtained, resulting in missing values. 
Honor Roll Performance (the number of times an 
individual recruiter appeared on the 
NAVCRUITCOM Honor Roll during the test period) 
paygrade; education; active duty (number of 
years); General Comprehension Test, Arithmetic 
and Mechanical Score (GAMX) - on Navy's 
enlisted entrance test; Sales Aptitude 
Battery/l6PF-m test score (SABX); rate; age. 
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Arima, James K. CApY il L376) 

To investigate and document Navy recruiting 

as a process that interacts with the larger 
military community of which it is a part and 
the civilian community which provides the raw 
materials it processes into accessions for the 
Navy. 

Interviews were conducted and operations were 
observed at every echelon of the Navy Recruit- 
ing Command (NAVCRUITCOM), Armed Forces 
Entrance and Examining Stations, the Armed 
Forces Vocational Testing Group, and the 
Recruit Training Command during the last three 
quarters of fiscal year 1974. 

This study focused on how the Navy Recruiting 
Command operates and made recommendations to 
improve the system. The only result identified 
that related to this thesis was that failures 
in recruit training were attributable to faulty 
selection by NAVCRUITCOM. Furthermore, goal 
accomplishment for the fiscal year was 
accomplished by failing to meet the quality 
standards that were in effect. The prevailing 
norm in the system was dedication to hard work 
in order to meet recruiting goals. 

Documents recruiter problems. 

No direct application to the problem of 
profiling the successful recruiter. 
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Method: 


LAGY Iva, R abe 

"WU. S. Marine #Corps KRecrul ters er to tee 
Predrechion aac. 

To apply the Navy" s recruiter selection 
battery €0O° Guiiameiiercorps samp ue 

The inventory was aquministered so allvactive 
aunty Lectrui Gets team, Ene JlCAmvaaine sGomu. 
district (N=470) and consisted of the same 
VCeMms USC hn eiemiecirg.( 19 2) stud) awe 
author analyzed the data using the formula 
developed by the Navy to predict recruiter 
success. 

The analysis indicated that the Navy formula 
was not a valid equation for predicting Marine 
Corps Yecrurveer (pmeducl ion Mowe VvVer mae! 
recruiting setting (rural-urban) and geograpini 
cal differences were taken into account in the 
Griterion;, relaeively successtwm prediceivon 
Formulas were Shermed? One Lor uEban recruited. 
and Gne [OR Swe pecCrultewo cr 0SS-Vvaligiag 
results suygest that this wethod may be a 
useful serecning device fomeeanqdidates for were 
Marine Corps YTeCcrurgeres ope 

The number of accessions of urban and rural 
TeECruUlkers Separately andweormeercd for 
geographical differences in relative perfor- 
mance of recruiters. 

Urban Urban lowe halter merit Ie ie lrow 
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Abrahams, N.M., Neumann, I 
Gro 11 Paeoe) Ss ) 


Paneer itMeana, 1B. 


Comey £or Selection of Navy Recruiters 

To @Puprovemreermiiter selection through the 
USewtot Ue mSErOnG VOCational Interest Blank 
Goi) eanad Obier predictor winstruments: 

SVIBS were collected from samples of the most 
and least effective recruiters at 36 of 42 


BEerulEIng Stations. "rhe responses of the two 
groups were compared for one-half of the sample 
and used to establish scoring weights. The 


valid responses were then assembled into the 
RPeGrulte ro iitemest Scale-] (RIS=-1). The other 
half of the sample were scored on the RIS-1 to 
determine how well the scale discriminated 
between the most and least effective re- 

Chur Ler s . 

MmewovIB Scale, Rils-1l, proved to discrimin- 
ate quite well between the most and least 
effective recruiters. When scores of the 
“hold-out" group were ordered, the top quarter 
had approximately three times as many effective 
Peeruiters aS Gilad tne Dottom ftourth. 
successful in cross-validation. 


Oe Bttecktiveness designations were made by 
Che Commanding Officer - subjective. 
b. The recruiters involved in the comparison 


represented the extremes in terms of effective- 
ness so the degree of discrimination achieved 
by the SVIB scale is greater than it would be 
in a sample representing the entire range of 
effectiveness. 

See Sc elminaty slic ncdamonmly, Lurther research 
is needed. 
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?) 


raf, RC ame ee een eo ae: (June “Laeioe 

The Development ob Aneincer es bal nvel Somos 
the Selection of Marine Corps Recruiters 

Using @ Versionmorm Che Navy kl Sam kecr meen 
Interest Scale), the MCRIS (Marine Corps RIC Pe 
the authors hoped to select Marine Corps re- 
cruiters. 

MCRIS; used above-average, average, and below 
average recruiters as their criterion measure. 
Higher validity coefficients than the Navy 
scale. 

Was not cross-validated; needs a more reliable 
metnod of measuring recruiter performance. 
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BOmMalts. W.Ce > ROSSS oR. 
(Sep. 1981) 


., and Touquam, J.L. 


Selene ied este eeigy 

To expand and refine the test battery and to 
delermmnmey TESevalidity Lor predicting various 
dames TONS BOL GECerUmGEer sper tormance — third 
phase. 

The Test Battery was revised by including 
additional experimental items selected on the 
basis of their hypothesized relationship to the 


wader ining eonstructs  TorL,ene battery. Admin- 
istered to 194 Navy recruiters in seven 
different locations. The two primary measures 


of success used were (a) production data 
compiled over a 6-month period, and (b) ratings 
gathered from supervisors and peers on four 
aspects of performance. Factor analysis was 
HSed Gomidenci fy van d cons ERUCtS. 

Composites of the new items successfully 
INeasunred timer Eargec Gonseructs. “In about 
half the cases, validity of the constructs was 
also improved. The scales derived from the 
GOMSErUcES Validiy predicted reeruiter 
productivity (average number of persons 
recruited) and rated recruiter performance. 

The primary purpose of this research was to 
develop measures which would predict Navy and 
Marine Corps recruiter performance. The pro- 
cedure used successfully identified personality 
and vocational interest constructs related to 
one or more aspects of recruiter effectiveness, 
and successfully developed additional parallel 
measures of these constructs. This study 
provides additional stability to the results of 
Cie wt tLSCeuEeWVONCt TORE om imeeiatesimllar solutions 
are found regarding selling skills, human 
relations skills, and organizing skills. 

Poeor analvVsis was USsedmanaderie Inter-rater 
reliabilities of the factor scores completed 
were .62, .48, and .65 respectively, which was 
Surtiecienctly highmeema stows tnewitactor scores Co 
represent individual recruiters! effectiveness 
in these three areas Of Navy recruiting. 
Response rate data were not available for the 
new items since the battery administered to the 
applicant sample did not contain these items. 
(The applicant sample consisted of 131 fleet 
personnel who had volunteered for recruiting 
duty and completed the same inventory battery 
that was administered to the previous con- 
Guerent validity sample.) 


ILIGre, 


Production 

Selling skilis, Human relations skills, 
organizing Skiilj@and overall pertemnaaiee- 
Performance was broken into two categories, 
personality items and vocational interest 


items. The constructs of the personality 
items are as follows: 
a. Selling skills: Good impression, impul- 


Sive, enjoying being center of attention, 
working hard. 
b. Human relations skills: People oriented, 
Spontaneity and impulsiveness, unhappy and lack 
of confidence, ambitious and working hard. 
Cc. Organizing skills: Order and planning 
ahead, leading and influencing others, unhappy 
and discouraged, "bad actor". 
d. Overall performance: Working hard, impul- 
Sive, leading and influencing others, good 
impression, people oriented. 
The COnstruetrs of the Vecaeronal interest 

items were: 
a. Selling skills: Extroverted interests, 
interests in detail work, law and political 
interests, sports interests. 
b. Human relations skills: Extroverted 
interests, interest in teaching, "feminine" 
interests, interest in newSpaper work, sports 
interest, religious interests. 
GC. Organizing GkKills: ~interestsin politics? 
interest in detail work, "feminine" interests, 
leadership interests. 
d. Overall performance: Law and political 
interest, extroverted interests, sports 
interests, interest in teaching, "feminine" 
interests. 

Tne validity of Einaimakevs for predictime 
production (N=194) were as follows: 

Correlation with 


Predictor Key Pmoduwet lon 
selling) Skis we2k 
Human Relations Skills 5 es 
Organizing Skills ae 
Overall Performance ~ 260% 

®p < .O1. 


110 


io. 


menor ; 
ie Le % 


Gl Sale 


Method: 


Sore :: 


Pe Ove 
Explan: 


Pew icer mn .©.  Abravans, .M., and Trent, T.T. 
(May 1986) 

Pelee Tomer ec ie witli ieomeCr Ds oOec lal Ass igqn—- 
iene cot eemy (GAB ) 

To establish objective procedures that would 
Mito vemEaiOmiarinewcore suabrlity Co select the 
NESoiMantled  Morines —cOourecrulting duty. 
Standard correlational techniques. 

Recruiters whose scores were in the top 20% 
obtained 27-40% more recruits than recruiters 
who scored in the lowest 20%. Results strongly 
confirm findings of developmental work. The 
recruiter selection composite is related to 
important aspects of recruiter performance. 
Production/SAB Scores 

aoe Nee wes  Hltalere bactons; Organlrzing 
Skills; Overall performance (664 applicants) 
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Author: 
DE lag ag) 


Method: 


Cone il: 


Borman, W. €or is (June 25333) 

"Recruiter Assessment Center: Candidate 
Materials and Evaluator Guidelines" 

This report contains materials and evaluation 
guidelines for a series of exercises to 
determine an individual's potential as an 
Army Recruiter. It outlines each of six 
methods and then explains how each are 
evaluated. The six exercises consist of: 
Personal Interview 

Cold) Calis 

Interviews with potential recruits 
Interviews with concerned parents 
Five-minute talk about the Army 

In-basket exercise 

Six exercises are designed to simulate 
Situations in which recruiters find themselves. 
The exercises also provide opportunities to 
assess an individual on 17 personal 
Characteristics. 

This report does not offer any results, only 
guidelines to which the recruiters were 
evaluated. 
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Borman, 4. 6 = (1982) 

"Validity of Behavioral Assessment for 
Predicting Military Recruiter Performance" 

To determine the validity of an assessment 
center designed to select U. S. Army 
rEeCruleers. 

Assessment Program which consisted of the 
following phases: 

a. Structured interview - targeted at sub- 
Ject's level of achievement motivation, 
potential for being a "self-starter", and 
commitment to the Army. 

Db. Cold calls - phoning three prospects for 
Ehe purpose of™getting them into the office. 
c. Interviews - Follow-up cold call and 
promote Army enlistment to two of three cold- 
call prospects. A third "walk-in" was also 
interviewed. 

d. Interview with concerned parent - interview 
with father of one of the prospects. 

e. Five-minute speech about the Army - 
delivered to the rest of the group and 
assessors. 

f. In-basket - subject was given an in-basket 
filled with notes, phone messages, and letters 
in which to take action. 

Subjects were 57 soldiers (all but one were 
em mentor Ing the Uo wmarmy recruiting school. 
Assessors were 16 experienced and successful 
Army recruiters. Validity of the assessment 
ratings was evaluated by correlating them with 
subsequent success in training. 

A composite of assessment ratings yielded 
corrected validities of almost 50%. "First 
impression" evaluations, ratings based on 
structured interviews, and test scores 
correlated near zero with performance in 
t€faining. Results of the study confirm that 
valid assessment does not require behavioral 
scientists as assessors, and analysis suggest 
Chat statistical composites of assessment 
ratings on individual exercises may be slightly 
more valid than "clinical" consensus judgements 
made after discussing assessment performance. 
Ratings for the speech, interview with 
parent, in-basket, and interview exercises had 
the highest validities. The structured inter- 
view was less valid than simulation exercises, 
and tests were also low in validity as well. 


orieS 


Seem 


Weak: 


The research design provided an opportunity to 

evaluate the following for success in training: 

a. First impression, physical attractiveness, 

and likability. 

db. Structured Titeryiew- 

c. Test scores On a persomaliey, inventory 

previously developed fo selectUerecruite., 

d. Subject "Ss pertemmance in individual 

SMS Lees. 

e. Consensus ratings of subjects reached after 

assessors discussed their performance each day. 
Also, there was virtually no chance of 

criterion contamination because the assessment 

ratings were completely independent of 

subject's performance during training. 


a. Small sample size. 
D. Costly. 
c. The range of assessment ratings was re- 


stricted since seven people who were evaluated 
during the assessment program, but not included 
in the walidityvanalysis because they dropped 
out before training tended to either have very 
high or very low assessment ratings. Also, 
four more subjects with low assessment ratings 
were eliminated because of inadequate 
performance. 
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"Predicting Army Recruiters' Job Performance 
From Development Center Ratings" 

ion reteecmElem rat lNGsouermene Ofl1Ginal Sample 
and a subsequent ‘development center' sample to 
a Measure of job productivity, the number of 
Contracts tne new recruiter produced in his/ner 
CieSEe year on the Job. 

PeseesOrs tated 37 peOrential recruiters on 
eleven personal characteristics, such as 

Per oVastveness, SOGlapllity, EClexibility, and 
practical judgement, as displayed in several 
different situational execises. Also used 
Seepwise regression. 

Assessment Center ratings had low correlations 
with job performance in a small sample and in 
the large sample, correlations were significant 
for a combined cold /call interview exercise 
and speech exercise with job performance. 
Productivity of the recruiter's battalion was 
the single most important factor in predicting 
his/her job performance. Speech and AFQT 
predicted approximately 2% additional variance. 
a. The recruiter development center should be 
upgraded to the quality of the original 
assessmene Center. 


b. Flexibility and sociability are not 
adequate rating dimensions. 
c. Sales training technologies would be most 


beneficial to improving the recruiter's 
effectiveness and motivation. 

The assessment center ratings showed some 
etlveyeas predictors Ofrerecruiters job 
performance. 

a. Although the assessment center sample had 
good quality rating, sample size was too small 
to generalize results. 

b. Development center ratings and training 
school grades were compromised by changes that 
took place in the operationalization of the 
assessment center 

Job Performance 

Raeings: Coldmed lig eineervicw, speech, 
Pibas~et, eOMposlte straining Grades: Written 
test, Phase II (performance), interviewing, 
ECleDnOnent ma OLnvermpredi ecors: Navy test, ARI 
BES toe s(s@claomrity) 5) GPl~ (cautiousness).. 
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Author: 
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Method: 
Cones: 


Bennett, J.T. and Haber, Shee (une 2) 
Selection, Deployment and Evaluation of Marine 
Recruiters 

To determine eie tactlors Chet inf lueneemiari ne 
Corps Fecrulbem. 

Multiple regression 


A. Urban and suburban recruiters enlisted more 
peuple per month than rural recruiters. 
ee Recruiters in their home state enlisted 


more than those stationed more than 500 
miles from home state. 

(a & b are cesults from high enlistment area) 
Those who feit recruiting duty was a 
financial hardship enlisted more people 
per month than those who did not. 

d. Recruiters with prior service as career 
planners were more productive than those 
Wil@r Gel GaeiOne 

(c & d are results from low enlistment area) 

Regression equations were not cross-validated. 
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Best, a B. and Wylie, W.J. (June 7 4 ) 

Using N weeecraber Serr ibutes and AtEltudes ; 
A a Reolovsis 

To Pre@iee recruiter perrtormance using 
Peerulrler (character vwSuLes . 

survey Anawysis, Multm®pie regressien analysis. 
Data was collected from a sample of 49 active 
U> S&S. Navy recruiters assigned within the San 
Preanciscomerecruiting District. Used survey 
Peery lews CO ldent itive attributes cl ef feermrve 
Becrulteraes, ALLEGE interviews, additionalaiwo- 
graphical, career, and educational data were 
obtained from the service records of recruiters 
who were interviewed. 

The most favorable aspect of HERI LISLIL IAG duty 
was independent duty. Least desired was public 
Bpeak ting -pmeOover One-third of the recruiters 
felt their particular stations were overmanned 
while an equal number considered the station 
morn dodlsmco be LOO mign. One-£ourtCh of the 
Recruiters were overweignt. 

a. Attempt was made to equalize percentage 
representation in the sample of each of s1x 
mones Withim the Gistriet. 

b. Clustered sampling techniques were used to 
decrease the physical variables into such 
AeoupIngs eas Geograpmuc Gitferences (coastal 
Vo elnl and) sCOmMmunNbeyw ey we (Urban, SuDWmeban, 
Bite! eand Stabion ss 1Ze. 

€. Identitied subjective attributes based on 
Survey interviews which could be beneficial to 
expert systems model. 


a. Failed in cross-validation. 

Pe oOOKedra tony omemreceriliing Gistrict,, not 
necessarily representative. 

¢. A control group data set was not available 
for cOmparative purposes. 

ad. Used Commanding Officer's evaluations of 
each individual's “effectiveness" on a scale of 


1 to 5 (5 being the most effective). 

Commanding Officer used his own definition of 
effectiveness. 

Command evaluation of each recruiter (CRUTVAL). 
a. The area the recruiter was from in terms of 
urban, suburban, rural (HOMAREA). 

Dende CAGE)? 

eae Ger score) (GCT) - 

Gd. Total years of active military service 
(eRSSVC): 

e. Proximity of HOMAREA to a major body of 
Wacer by distance: (1) Less than 20 miles; 
Crye2OMtLoeZzuCeme les: (syaGreater than Z00 

miles (HOMPROX). 
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INDICES, VARIANCE, AND MEAN SQUARED ERROR 


Expert #1. 

Fidelity Standards Discrim. Variance Mean- 

Index Index Index Explained SG las 

Overall 
Profile Chl Bes 6505 ATS gue. 54 Leos 
Comm. 
Skills Ba. Z 66.5 41.8 ia 8 6 4.97 
Demog. 
Charac. I 8 ane FES aoe | 3) RAR se La 2 
Military 
Backg. 89.6 eae 226 60.37 2450 
Person. 
Charac. el a 74.0 2 See 75.96 3. G 
Behavior 
Charac. Boley 3.0 ZO. 3 66.79 3,10ul 
Specific 
Exp. Lawes Tacs 1B) 5 8, 60.65 2 ie 
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Indices, Variance, and Mean-Squareéed Error 


Peooert #2. 
Fidelity Standards Discrim. Variance Mean- 

Index Index Index Explained Sumer © r 
Overall 
Profile Sule eh 80.0 78.6 74.18 9.98 
Comm. 
Skills SB .2 56.0 Soy 18) 77.89 Lies 
Demog. 
Charac. ae.) Sue) To 4 95.16 AGS] 
Military 
Backg. Sly ara 244 Plies 9 952459 eat oet) 
Person. 
Charac. 7.0 To .3 99.8 8 49 Sais) 
Behavior 
Charac. 92.0 O15 G27 84.65 POSS 
Specific 
Exp. Olga: 3328 70.4 92.39 ts) 5: 
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Indices, Variance, and Mean-Squared Error 


Exper: #3. 

Fidelity Standards Discrim. Variance Mean- 

Index Index Index Explained SG eee 

Overall 
Profile 89.4 99.3 55.9 80.03 G22 5 
Comm. 
Skills 94.5 A3a5 SG 3S) AS 6.66 
Demog. 
Charac. o>. 0 49.2 92.1 90.40 ‘PSs 
Military 
Backg. je eee 59.4 34 83.08 3 ecu 
Person. 
Charac. 95.4 67.6 Bt 97.18 608 
Behavior 
Charac. 9329 65.0 5 eo ag 88.18 4.600 
Specific 
Exp. 89.4 Se 4 o a hae ps 80.00 1.94 
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Indices, Variance, and Mean-Squared Error 


Expert #4. 
Fidelity Standards Discrim. Variance Mean- 

Index Index Index Explained Sine © 
Overall 
Profile Ses Bo) Sav SES) oe 86.19 Gc 
Comm. 
Skills 90.1 A920) Site) 10 el 1 Lye 3) 3) 95.0) 
Demog. 
Charac. 75.4 48.6 1 0n3 566 6 Lie 
Military 
Backg. SiO pes real AS ale dior One 85:0 
Person. 
Charac. Con) 8 O20 ok oS) 7 eno.0) BS bate) 8 6 
Behavior 
Charac. (ois Sees, 59e 0 50.4 SOR 20 Seal 
Specific 
Exp. 916256 Cs) 68 93.48 Ses) S 
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The Expert Systems: 
Indices, Variance, and Mean-Squared Error 


Boxer te on 

Pidelity Standards Discrim. Variance Mean- 

Index Index Index Explained Sd iat 

Overall 
Profile el eS) a0) 5 10 47.7 Coo 9 ono 
Comm. 
Skills 7324 66.0 HS So ea6 Coe 
Demog. 
Charac. 2h 8} 4 aU al0 era Se 0 Omeeie) 
Military 
Backg. Sisal 20.0 66.3 93:..64 4 eo 
Person. 
Charac. 9433 oe 15.4 elisin s)4| 6220 
Behavior 
Charac. SiS Bas) 64.0 50.2 = Ro MeRS Je. 2 
Specific 
Exp. 956-..5 a are Diowe2 I eel tgs 36m 
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Bxepert #6. 

Fidelity Standards Discrim. Variance Mean- 

Index Index Index Explained SO oy erg 

Overall 
Profile es, SG 62.2 GeO 7 iP a6 
Comm. 
Skills 6.0 42.4 (sees. Sale) ooo 
Demog. 
Charac. 964 Spl s 64.4 5300 426 
Military 
Backg. 83% 2 40.6 6b44 79.74 eyes) 
Person. 
Charac. 96 5) a8) Sone > Sa 50 4.27 
Behavior 
Charac. Siem SOR SSG Gro S43 
Specific 
Exp. 925 pach 62.4 oe ee ple Die oun 
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The Expert Systems: 
Indices, Variance, and Mean-Squared Error 


Expert #7. 

Fidelity Standards Discrim. Variance Mean- 

Index Index Index Explained SG wrens 

Overall 
Profile 95.9 5) lk, 2 43.7 9271038 Sure 
Comm. 
Skills Bens 42.6 68.8 76.73 8.29 
Demog. 
Charac. 93.6 45.0 SD ey Ta oe | 
Military 
Backg. SiG. 6 B95) 4 8 Oey 5 3) 15.04 6.99 
Person. 
Charac. 90.8 Siowso Bae 82.48 6.03 
Behavior 
Charac. 92.8 5 596 3oer 86.24 3.68 
Specific 
Exp. 93.4 48.9 Zen tT oS, 2. 9e 
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Expert #8. 

Fidelity Standards Discrim. Variance Mean- 

Index Index Index Explained SGnae nm 

Overall 
Profile py el Tae) 83.8 BS a2] Seo 
Comm. 
Skills Bee] 45.0 Ta. 16293 oats lal 
Demog. 
Charac. a6 42.2 57. 8 87.63 5.09 
Military 
Backg. a 2 3529 S259 83.85 oe! 
Person. 
Charac. Tis) ae 67.0 G4 61-533 126s 
Behavior 
Charac. 82.4 81.6 92.4 Ge 309 
Specific 
Exp. 92 67.2 S15) ra SL Go0 1 5.43 
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Expert a9. 

Fidelity Standards Discrim. Variance Mean- 

Index Index Index Explained > Gir 

Overall 
Profile Sase3 62. 0 69.9 69.43 9.66 
Comm. 
Skills Soo Srey O 86.0 T7252 LO wars 
Demog. 
Charac. O stoheeall SB ol OZ eee Gleesur4 Perdis 
Military 
Backg. 90.4 BES 5s. 94.8 81.86 10.40 
Person. 
Charac. ‘mee 65200 92.7 56.90 157 
Behavior 
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Specific 
Exp. Iie Zoe 8 S459 S65 0)/ lage 
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The Expert Systems: 
Indices, Variance, and Mean-Squared Error 


Expert #10. 

Fidelity Standards Discrim. Variance Mean- 

Index Index Index Explained Sq. Or Fr 

Overall 
Profile 9.52.4 Galiy.0 S90 Jie 3 Geo 3 
Comm. 
Skills 94.6 64.0 fess) 89.49 6.47 
Demog. 
Charac. Ss) eas: pa) SG.) SBeers 4.36 
Military 
Backg. Bel s.0 Deas 68.6 SiZeagio finelie' 
Person. 
Charac. O41 ie os LUO. 3 OOM OL oe 0 
Behavior 
Charac. siorn0 66.5 84.4 Oi 2 Sele) 
Specific 
Exp. 916555 48.4 Uk 6 se rr A | 4.60 
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The Expert Systems: 
Indices, Variance, and Mean-Squared Error 


ieesc fa lL. 

EF idegmey Standards Discrim. Variance Mean- 

Index Index Index Explained SG) Gale 

Overall 
Profile 26,7 (Orde: 76.0 75.34 9.44 
Comm. 
Skills Saliva) 38.3 124.2 66.88 li vey 
Demog. 
Charac. 95.8 3278 745 971282 oro 
Military 
Backg. Seg s 48.4 78.1 Oa 6 See 
Person. 
Charac. sia 5 0) 3) (Ab 95740 72.40 12344) 
Behavior 
Charac. 95.1 63.0 eS 90.57 6.09 
Specific 
Exp. 97.0 ens..3 85.4 94.25 5 LZ 
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The Expert Systems: 
Indices, Variance, and Mean-Squared Error 


Mxpert #12. 

Fidelity Standards Discrim. Variance Mean- 

Index Index Index Explained Sela deg 

Overall 
Profile Bee. 0 ies POS. 8 90.38 8.05 
Comm. 
Skills 96.6 6.9 S03 93.37 iS) oleh 
Demog. 
Charac. oe 7 Be 58.9 86.00 Sue Syl 
Military 
Backg. 98.6 23.0 70.5 97 243.2 2.89 
Person. 
Charac. 98.0 30.9 Nee’) 96.13 6 UN 
Behavior 
Charac. 96.0 20 101.4 92.34 (uy 
Specific 
Exp. 9 4en3 11.9 714.3 88.99 6.16 
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The Expert Systems: 
Indices, Variance, and Mean-Squared Error 


Bxpere stl3. 

Fidelity Standards Discrim. Variance Mean- 

Index Index Index Explained S Ciskii 

Overall 
Profile 96.0 5 oe G2 oe 92.16 4.36 
Comm. 
Skills 96.7 64.0 Sy ae il 93.59 3.93 
Demog. 
Charac. Ssh 4 6235 46.9 96.48 2. 
Military 
Backg. 89.4 BIT) Ate 4623 80.00 5 Sige 
Person. 
Charac. 973 oe 66.0 9 4e7GsS 3: 
Behavior 
Charac. 95.9 68.0 Sse (0 92.11 4.41 
Specific 
Exp. 9: SamzZ 49.2 5 3a 90.78 4.03 
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The Expert Systems: 
Indices, Variance, and Mean-Squared Error 


Expert #14. 
Fidelity Standards Discrim. Variance Mean- 

Pade x Index Index Explained Seiten 
Overall 
Profile Gigw. | 427 Se 3 8 ie 9.50 
Comm. 
Skills S35. 9 G24 64", 5 13385 Soy 
Demog. 
Charac. 96.0 82.0 114.8 92.29 820 Z 
Military 
Backg. 621 41.6 49.5 47.83 8.94 
Person. 
Charac. 85.6 TCG. 3 o5..0 Tse S| Livres 
Behavior 
Charac. age: 6 103947 67.2 6545 LO Z0 
Specific 
Exp. hic) Silas 62.4 74.36 7.91 
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The Expert Systems: 
Indices, Variance, and Mean-Squared Error 


Expert #15. 

Fidelity Standards Discrim. Variance Mean- 

Index Index Index Explained SG eas 

Overall 
Profile 9431 670 shits Jl Soe a7 > 
Comm. 
Skills SH eee | 60.6 8057 82s 8.48 
Demog. 
Charac. 93.4 76.6 5925 et fees (0) Dee 
Military 
Backg. Se ees as 5 5 S377 Oa. 14 a2 28 
Person. 
Charac. 93°20 69.3 1 oe 86.66 eae 
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Indices, Variance, and Mean-Squared Error 


Bxpert #16. 


Fidelity Standards Discrim. Variance Mean- 
Index Index Index Explained Sep ear 

Overall 
Profile 92.6 60.7 92.4 85.83 8.65 
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The Expert Systems: 
Indices, Variance, and Mean-Squared Error 
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Fidelity Standards Diserim.  Vanritanee Mean- 
Inde Index Index Explained > Cine 

Overall 
Profile oes ms > AOS 94.78 yee hes: 
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Demog. 
Charac. 96.6 3) Se os wee 93.47 Bar fies 
Military 
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Specific 
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Bxpert MR 
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Index Index Index TCO Ls gvena SHS ed) ong 

Overall 
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